Topic: Exporting millions of records
Hi,
I suspect no-one else has this problem but I may be wrong? I need to export 16+ million moth records, the entire National Moth Recording Scheme database, into a single file for a data requester. The Report Wizard won't handle this much data in one go, having tried on numerous occasions. I can create numerous WZD files and run these but this is unsustainable in the long run.
What I do have is an excellent SQL Server Integration Services (SSIS) batch utility written by Dr Stuart Ball which exports the whole database in the NBN Gateway format but this is of no use to a data requester as all it exports is a table of keys. Is there such a batch utility in existence that will do a similar job but with meaningful, user-friendly data?
In the meantime, I will look at redeveloping Dr Ball's utility to see if I can get it to work as needed.
Best wishes,
Les
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme