|
The technical forum of the NBN Trust for use by the Network. It includes discussions and announcements on the NBN Atlas, Recorder 6 and Indicia |
You are not logged in. Please login or register.
Forum → How Do I... & Other Questions → Negative number of Records Imported to Recorder 6 ?
Importing data from Recorder 2002 to Recorder 6 via nbndata zipped file. The final "Import Complete" message tells me that "-2" items were successfully imported. Can anyone tell me what a negative number of imports signifies please.
The fact that is negative is definitely a bug in Recorder - which version are you using? I seem to remember seeing this in some older versions. But the other issue here is that your import probably failed and nothing was imported. Did you get any validation failure messages?
Best Wishes
Hi John,
Thanks for the response.
We are using Version 6.13.2.176
The import was considerably reduced in size to try and track down a specific record generating an import error, so majority was invalid.
It would be nice to know how the field goes negative, but on the assumption it is a calculated field, and the only reason we suspected the figure was that it was negative, how do we know it is accurate when the figure isn’t suspicious?
Regards, Dave
We've done a huge amount of imports here on just about every version of the import wizard and have never seen a bug like this - it has always been accurate. I would be a bit concerned, to say the least, if I got a negative number back.
Hi Charles,
Appreciate the comment, but if your input is nbndata zipped (content unknown), how do you know your import was accurate. You have no start point, only what Recorder says it has done.
Cheers, Dave
The NBNData (zipped) format is just an Access database (MDB) within a zip file. You can simply unzip the MDB and take a look inside. If you look in the TAXON_OCCURRENCE table you can check to see if this tallies with number of occurrences that get imported.
I investigated this problem some time ago and found that imports delete taxon occurrences that don't have taxon determinations. Therefore if you import a file that contains less records than you have occurrences without determinations you will end up with a database containing less observations than you started with. Personally, I don’t approve of software tidying things like this up without telling the user.
Without determinations these occurrences are unusable as there is no species name to display in the observation hierarchy or in reports but you may like to look at the data in the occurrence, sample and survey event to investigate further. I have suggested to JNCC that if an observation doesn’t have a determination or it has one without a valid taxon list item key that it should be displayed with ‘Taxon name missing or not in dictionary’ in place of the taxon name in the observation hierarchy and reports so that these items can be investigated/deleted easily via the user interface. However, we may be unlikely to see this unless a number users endorse it. I think this situation can arise when Recorder crashes during data entry.
We could really do with a suite of XML reports that look for problems like these to give users an opportunity to correct them.
Thanks for pointing this out to me Sally, but I'm afraid I don't really understand what you mean by "if you import a file that contains less records than you have occurrences without determinations." How can this be? The import file has x records in it and recorder generates the determinations during the import. Have you mistyped this sentence or am I missing something more profound?
With reference to my other post:
http://forums.nbn.org.uk/viewtopic.php?id=1397
You will see that all the records I imported effectively had invalid determinations, but the database did not end up with less occurrences in it after the import, otherwise it would not occur when I tried the import the second time, or the third and it did.
Rob,
As I understand it this would be a difference between an import that had been exported from another recorder database and importing from an Excel or similar file. In the first case there would already be determinations which should come in with the import, but if some were missing it would cause the problem Sally mentions. In the second case the records are not in a Recorder specific format and the determinations would be created as you perform the import.
Hoping I haven't got the wrong end of the stick.
Gordon
I should have qualified the 'occurrences without determinations' with say, ' already in the database you are importing the data into'. If Recorder 6 crashes when doing data entry using one of the options on the Data Entry menu I have seen it create the survey event, sample and taxon occurrence but not the determination - see http://forums.nbn.org.uk/viewtopic.php?pid=4539#p4539 . If this has happened several times, your database may contain several taxon occurrences without determinations. If you then import a file containing a small number of observations, i.e. fewer than the number of taxon occurrences without determinations, the import will delete the occurrences without determinations and import the observations in the import file and report that it has imported a negative number of observations.
Taxon occurrences without determinations are a minor form of corruption which users should tidy up. Hence my comments that ‘Taxon name missing or not in dictionary’ should be displayed when appropriate and that we could really do with a suite of XML reports that look for problems like these to give users an opportunity to correct them. I have also seen occurrences with more than one preferred determination and occurrences with determinations but without a preferred one, neither of which are valid. Samples without occurrences may be something that should be tidied up as well although probably not survey events without samples. In Recorder 3 to Recorder 6 data transfers, R3 events are transferred as survey events with nothing in them.
Without doing the tests on your database myself, I can't explain your point that 'it would not occur when I tried the import the second time, or the third and it did' as I don't know exactly when it does the deletion.
I understand that now Sally,
but as far as I can tell nothing was imported because the determinations could not be created legitimately. There were no occurrences without determinations before I did the import (or at least I have no reason to believe there may have been.
My problem is simply a reporting error, no changes to the database having taken place.
Well, I'm still confused :rolleyes: and I'm getting negative records with a spreadsheet import:
[img]http://forums.nbn.org.uk/uploads.php?file=rec-bug.PNG[/img]
So has Recorder deleted the invalid occurences?
And Recorders failing to insert a biotope occurence... even though I haven't asked it import any?
Failed to insert SR0003510000018W into BIOTOPE_OCCURRENCEIf your situation is the same as mine Charlie, then I doubt if Recorder imported anything at all.
It certainly shouldn't have imported anything invalid (but who knows?)
As for the biotope, do you have any records with associated habitats, or substrate data in?
If your situation is the same as mine Charlie, then I doubt if Recorder imported anything at all.
Hmm... well it looks like it has :rolleyes:
As for the biotope, do you have any records with associated habitats, or substrate data in?
Not that I'm aware - it was just a plain spreadsheet with species, date, location, grid reference :/
Charlie,
What sort of other errors did you get in the log? Or was it all to do with biotope occurrences?
Detecting occurrences with missing determinations can be done with the following query:
SELECT O.*
FROM Taxon_Occurrence O
LEFT JOIN Taxon_Determination D
ON D.Taxon_Occurrence_Key=O.Taxon_Occurrence_Key
WHERE D.Taxon_Determination_Key IS NULLFeel free to wrap that as an XML report if anyone has the time. But In your case I wonder if something else is going on, as the biotope occurrence thing does sound odd.
Forum → How Do I... & Other Questions → Negative number of Records Imported to Recorder 6 ?
Powered by PunBB 1.4.6, supported by Informer Technologies, Inc.