1 Reply by martinsanford

  • martinsanford
  • Veteran
  • Offline
  • From: Suffolk
  • Registered: 02-11-2007
  • Posts: 52

Re: Problems with large exports

I'm having problems exporting large amounts of data. Using the report wizard I've selected 130,000 records I'd like to export in NBN zipped format. When I try and export them the system goes through the external validation checks OK but hangs during the process 'gathering information for export' getting stuck at different stages Sample Recorder, Ref Author or Sample Sources. When I cancel the export and go to close Recorder there is a bug report due to an EOleException - exception message : The EXECUTE permission was denied on the object 'usp_Setting_Update', database 'NBNData', schema 'dbo'. I'm not sure if this is due to my cancelling the process or if it is the cause of the export failure.
I have tried selecting small batches of data (<1000 records) and these seem to export OK.
Any help or advice gratefully received
Martin

martinsanford's Website

2 Reply by Lynn Heeley

  • Lynn Heeley
  • Administrator
  • Offline
  • Registered: 03-07-2008
  • Posts: 345

Re: Problems with large exports

Hi Martin,

Which version of Recorder are you using and which version of the NBN exchange addin?

Cheers,
Lynn

3 Reply by martinsanford

  • martinsanford
  • Veteran
  • Offline
  • From: Suffolk
  • Registered: 02-11-2007
  • Posts: 52

Re: Problems with large exports

We've upgraded to latest versions: Recorder 6.14.4.212
We've got the NBNExchange addin version for Rec 6.13

martinsanford's Website

4 Reply by wilkin_s

  • wilkin_s
  • Administrator
  • Offline
  • Registered: 31-03-2006
  • Posts: 87

Re: Problems with large exports

Hi Martin
Can you check the smaller version point number of the addin please? To get this right click on the NBNExchange.ocx file and go to the version tab. I think the latest version is 6.13.2.24.
Secondly the export can take a bit of time. I would suggest leaving it running for a good bit - how long have you left it before terminating it? Is the system still busy (eg. is there processor activity or disc activity?
Can you also just let us know how many records are in you system at the moment and whether you are running on Express or full SQL Server.
Best wishes
Steve

5 Reply by martinsanford

  • martinsanford
  • Veteran
  • Offline
  • From: Suffolk
  • Registered: 02-11-2007
  • Posts: 52

Re: Problems with large exports

Yes exchange addin version is 6.13.2.24. Note the problems seem to be with exporting to NBN Access Zipped file, I haven't tried the NBN Exchange format yet. As there is no progress bar for this part of the export its difficult to tell if its doing anything, I've tried leaving it for about an hour before cancelling.
We're running on SQL Server Express 2005 and have 1,013,768 records in Species obs. I'm hoping we'll be able to export all these to NBN Gateway once I've resolved all the annoying validation issues triggered by det dates given as a year appearing to come before precise sample dates.

martinsanford's Website

6 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Problems with large exports

Hi Martin,

I think you are just hitting a performance limit of the process which exports data, as 130,000 records is quite a lot to export in one go. You could either try leaving it overnight, or perhaps try exporting in smaller chunks?

Best Wishes

John van Breda
Biodiverse IT

johnvanbreda's Website

7 Reply by martinsanford

  • martinsanford
  • Veteran
  • Offline
  • From: Suffolk
  • Registered: 02-11-2007
  • Posts: 52

Re: Problems with large exports

Just tried again using a filter with a bounding box rather than the Report Wizard and polygon to select the records. 'Export using a filter' was wable to export all 130,000 records in about 40 minutes

martinsanford's Website