1

Topic: Maximum number of records on Import?

Hello,

We've been having a few Importing 'issues' here at LRERC on Recorder 6.22.2 and wondered if anyone could give us some advice; we're currently trying to work out if the problems are in Recorder, or with our Server.......

In theory, does the Import Wizard have a maximum number of records that it can process in one Import?
Is there a 'recommended' maximum number of records per Import?
Is there another way to add records to Recorder (as a large number, not via the data entry screens or using Recorder to Recorder exchange)?

Thanks

Kirsty

2

Re: Maximum number of records on Import?

How many are you trying in one go? I can't remember our maximum but I'm happily doing 25,000 records at the moment. However, when we were connected to a cloud server, that number would have been impossible because our internet was so slow - one of the reasons why we brought the database in house.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

3

Re: Maximum number of records on Import?

Thanks Charlie - we're stuggling with more than about 10,000 at the moment (on our in-house server) - it doesn't seem to be a consistent problem and so are trying to work out what the causes may be.  Possibilities are different results on XP/Windows 7 (we're currently running the same networked copy of Recorder over both operating systems), or possibly server capacity.  However, we thought it might be best to try and get a feel of other experiences and any 'best practice' that's around as well as running test imports on our own system.

4

Re: Maximum number of records on Import?

Hi Kerry,

I regularly import 75,000 records at a time and have done for years now. I use the ADO Connection to import from MS-Access.

I use latest Recorder 6/Dictionary, W7, SQL Server 2008, Windows Server 2008 R2 Std over a virtual network.

Les Evans-Hill
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme

5

Re: Maximum number of records on Import?

Is there a specific error message?

Do you have anything other than basic "who where what when" columns matched? There have been problems with complex 'abundance' values in the past I think.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

6

Re: Maximum number of records on Import?

What exactly is  happening when you do the  imports. Is it just hanging ?

Mike Weideli

7

Re: Maximum number of records on Import?

Thanks All,

To try and answer your questions (apologies as we are not the most technical of people here....)

There are no error messages, it just hangs for a daft amount of time.  Sometimes it can be that we can try the import again instantly and it goes in no problems.

We've been trying very simple imports due to the work of a certain project containing only taxon name, determiner (with a single value), recorder (again, a single value), location name, grid ref, date and taxon comments, so nothing too odd and considerably more basic than out 'usual' requirements for our data standards.

We're spending today attempting to do some more detailed testing (a mix of import sizes and using XP and W7) to try and establish if we seem to have a 'maximum' and if it varies per operating system.  It may be that these results will allow me to sound slightly more intellegent!

Charlie - how long is the 25,000 records taking?  Is the source Excel?

8

Re: Maximum number of records on Import?

As an XLSX 25,000 records takes (excluding location matching!):

2 minutes to process the XLSX
20 minutes to validate the columns
5 minutes to generate the import file
5 minutes to import the file

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

9

Re: Maximum number of records on Import?

Thanks Charlie,

That's roughly the same as we were achieving yesterday, with 31 mins start to finish for 20,000 records (also excluding location matching!).  However, 3 weeks ago this was taking 15-20 mins longer, if it was working at all.....

Here's hoping that our ICT people have tweaked something behind the scenes following the conversation that happened at that time!

Kirsty