1

Re: Import Rejects confusing

Dear All,

The other day I exported some data for one of our outstation sites and imported the data into their copy of Recorder 6.

It then ended saying there were some import rejects. When I looked in the ImportRejects.txt file I see the following confusing (to me at least) information:

Items rejected by Recorder 6 in data imported at 16-13-14 on 27/11/2007:

Failed to insert WA00003500000Q9J into TAXON_OCCURRENCE
Database Error Message:
INSERT statement conflicted with COLUMN FOREIGN KEY constraint 'FK_TAXON_OCCURRENCE_SAMPLE'. The conflict occurred in database 'NBNData', table 'SAMPLE', column 'SAMPLE_KEY'.The statement has been terminated.
--------------------

I guess this is pretty standard information, but I have not got a clue what it means or whether it means my data has not imported correctly and what to do to correct it.

Does anyone have any idea on what it all means please and whether I have a serious problem going on in the database?

There are around 59 of these statements related to Taxon_Occurrence, Taxon_Occurrence_Data, Taxon_Determination and Name_Relation

Many thanks

Brian

Brian Miller
(Conservation Officer (Buckinghamshire), BBOWT)

2

Re: Import Rejects confusing

Brian, this means that your import file contains a taxon occurrence that is missing its parent sample. This "foreign key constraint" means that the value in TAXON_OCCURRENCE.SAMPLE_KEY (which is a foreign key) must first exist in SAMPLE.SAMPLE_KEY (which is a primary key) before the insert will execute.

Have a look in the export file for the taxon occurrence and check to see if its parent sample really doesn't exists. If it doesn't, then that begs the question: how did a bunch of occurrences get exported without their samples?

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

3

Re: Import Rejects confusing

Charles,

Thanks for reply. In this case the Taxon Occurrence parent sample key does exist. It is listed in the Taxon Occurrence table and also then in the Sample table.

Following these errors through in the same process, the failure then to insert into Taxon_Occurrence_Data table the TOD_Key exists in the T_O_D table with the T_O_Key next to it corresponding to the error one in the T_O table.

The same is true of the next ones of Taxon_Determination.

So far in these there were 7 errors in each section.

Next I get to the Name Relation insert failure, and I now start to remember a similar problem I had some time back... but continuing here:

There are 37 with the COLUMN FOREIGN KEY constraint of 'FK_NAME_RELATION_NAME1' and one of COLUMN FOREIGN KEY constraint of 'FK_NAME_RELATION_NAME2':

Failed to insert WA0000440000000C into NAME_RELATION
Database Error Message:
INSERT statement conflicted with COLUMN FOREIGN KEY constraint 'FK_NAME_RELATION_NAME2'. The conflict occurred in database 'NBNData', table 'NAME', column 'NAME_KEY'.The statement has been terminated.

Looking this one up I discover that in the Name_Relation table there is a N_R_Key of WA0000440000000C, a Name_Key_1 of WA0000440000000L and a Name_Key_2 of WA0000440000000G. The N_K_1 is listed in the Name table and is marked as an organisation (-1). Is this possibly the route of all errors? However, sorting the Name_Relation table on Name_Key_1 I do discover the WA000044...L is there again, but clearly with no errors related to it, indicating the successful import of that... (can not check this now as do not have the other database here)...

Does this make any more sense to anyone please? I do remember some issue of organisations and Taxon Occurrences roled over from old Recorder 2000/2?

Thanks

Brian

Brian Miller
(Conservation Officer (Buckinghamshire), BBOWT)

4

Re: Import Rejects confusing

Hi Brian

Apologies for not responding to this sooner - has been on my ever-expanding 'to do' list!

The NAME_RELATION table is used to associate an individual with an organisation, or less commonly to associate an individual to another individual, or organisation to another organisation. The association is created by the NAME_KEY_1 and NAME_KEY_2 fields.

I think the first thing I would check here (if you haven't already done so) is that all the Name_Keys in both the NAME_KEY_1 and NAME_KEY_2 fields exist in the INDIVIDUAL or ORGANISATION table in import file (or indeed in your copy of R6).

The second thing is to check that all Name_Keys in both the NAME_KEY_1 and NAME_KEY_2 fields exist in the NAME table in the import file.

Please let me know if you have already resolved things or if you've tried the above and still had no luck.

Many thanks,

Sarah Shaw
Biodiversity Information Assistant
JNCC

5

Re: Import Rejects confusing

Hi Sarah,

Thanks for getting back to me on this. I had a call from Sally about it and thought I understood, however, having gone through this checking process I do think there may still be an underlying problem going on.

Starting with the easy one:
The original database (where data exported from), all keys match between the N_R table N_K_1 and N_K_2 and the Individual, Organisation and Name table name keys.

When I do the same queries in the exported file (import file) I see the following results based on DISTINCT keys:

N_R table N_K_1 - only 2 match out of 53 (1 individual and 1 organisation)
N_R table N_K_2 - 98 match, but 2 do not (2 individuals not match)

Looking at the exported file and comparing the Name_Relation table - N_K_1 and 2 to the Name table I see consistent results to the above in that:

N_K_1 has 51 non-matches and N_K_2 has 2 non-matches.

The fact that all these name keys exist in the correct places in the main database indicates to me that the exporting is potentially missing off certain associations between records(?)

Not sure if any of this helps...?

Thanks

Brian

Brian Miller
(Conservation Officer (Buckinghamshire), BBOWT)

6

Re: Import Rejects confusing

Hi Brian

Many thanks for the above information.

I think the export/import functionality in Recorder 6 does need reviewing generally in terms of related data - that's not to say it is definitely misbehaving now, but it is worth checking what was originally specified and what the user requirements are. We won't be able to make any changes in this area for the next version, but I will put it on my to-do list to look into this further.

Best wishes,

Sarah Shaw
Biodiversity Information Assistant
JNCC

7

Re: Import Rejects confusing

Hi Brian

Just to provide an update on this - I spoke to Sally Rankin about this recently and Sally tells me that it was previously decided that any missing entries from the Name_Relation table that weren't associated with the actual data in the export (i.e. were not recorders or determiners etc.) would not be included in the export file.

It was decided to keep the Name_Relation table in the export - so any relations with the individuals/organisations already included would be kept, but that including additional individuals/organisations just for this purpose was perhaps a step too far - as potentially this could cause a high number of additional entries to be exported.

Hope the above makes sense. I would still like to review the export/import of related data at some stage anyway,

Kind regards,

Sarah Shaw
Biodiversity Information Assistant
JNCC