1 Topic by RobLarge (edited by RobLarge 24-02-2015 12:49:11)

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Topic: Stratiotes aloides

Here's an interesting one for you Chris, although it has been rumbling around for a few years and was mentioned here in 2011.

We have been attempting to submit a dataset to the NBN for a while and keep getting sent back (thankfully successively shorter) lists of problem records. Now bear in mind that these are records which have been submitted to NBN before without issue, but which are now apparently unacceptable. I have no idea what has changed.

The particular group of records which are causing the problem are all of Stratiotes aloides (Water soldier). Dario and Graham both insist that the TVK is invalid. All have the TVK NHMSYS0000345863.

Graham French says that "The TVK NHMSYS0000345863 for Stratiotes aloides  was removed from the UKSI back in 2010 and replaced with NBNSYS0000002110, which is the one to use"

But here's the thing, NHMSYS0000345863 is the TVK associated with this species in the VASCULAR PLANTS list (the preferred list), while NBNSYS0000002110 is the TVK which appears in the Recorder 3.3 list (not a preferred list and deprecated here).

Interestingly the Taxon_key associated with both TVKs is also NBNSYS0000002110. Coincidence, or just the cause of the confusion?)

Obviously the easy fix for us would be to redetermine the affected records against the appropriate TVK (we have additional records which are already determined in this way). However this would mean associating records which are currently determined against the preferred list, with a list which is not preferred. This cannot be right surely.

This looks to me like UKSI removed and retained the wrong TVKs, so how do we make this right (and what were they doing removing TVKs which are in use anyway?)

I wonder if this also applies to any of the other species mentioned in this thread http://forums.nbn.org.uk/viewtopic.php?id=4306

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

2 Reply by Gordon Barker

  • Gordon Barker
  • Veteran
  • Offline
  • Registered: 15-05-2006
  • Posts: 344

Re: Stratiotes aloides

I have had this problem before http://forums.nbn.org.uk/viewtopic.php?id=5689

Gordon Barker
Biological Survey Data Manager
National Trust

3 Reply by RobLarge (edited by RobLarge 24-02-2015 14:34:27)

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

In the light of Gordon's post it looks like R6 is serving up the wrong TVK then?

Can we have it fixed this time, four years is enough I think.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

4 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

TVK NHMSYS0000345863 is no longer in the master R6 dictionary, which leaves the question why have you still got it ?  Deleting TVK's is not a good idea, because it is easy to miss something in a related table which can stop it being deleted, but I would expect even  in this situation for the Taxon_List_Item to have been updated so that the TVK  was no longer in use.


I have lost track, but didn't we do a recent check and update on your dictionary ?

Mike Weideli

MikeWeideli's Website

5 Reply by RobLarge (edited by RobLarge 24-02-2015 15:58:56)

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

No, I don't think we did. But like you I find it hard to keep track.

Doesn't look like I'm the only one who has it though.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

6 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

OK I will go back through and try and see where the change was made and if this  throws any light on what might have gone wrong.

Mike Weideli

MikeWeideli's Website

7 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

The change was made in Dictionary Upgrades 0000000S and T on 21/09/2010. This is pre upgrade kit Version 4. It is possible that Upgrade R, S and T were run together instead of R being run first then S and T  as required at the time. If this happened then upgrade S would not have been processed.  T which does the deletions wouldn't have worked because the TVK was in use.

Mike Weideli

MikeWeideli's Website

8 Reply by RobLarge

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

OK so what do I do to fix it? I think I still have upgrade kit 3.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

9 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

The only way this can be fixed with any certainty is for me to give you a  special upgrade to bring your dictionary in line. To do this I need a copy of your database. A backup will do. What you get back is a  set of special dictionary upgrades, probably 2W,2X,2Y which you run in the normal way (Please get Upgrade kit 4 though).  You can carry on using Recorder in the normal way while I am doing the processing.

I will be doing the work under my JNCC contract, which will guarantee confidentiality of your data.

Mike Weideli

MikeWeideli's Website

10 Reply by RobLarge

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

It's OK, we do have Dictionary kit 4, I just meant that we still have kit 3 on our system.

How will I get the backup to you? They are currently about 3.4 GB in size.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

11 Reply by Gordon Barker

  • Gordon Barker
  • Veteran
  • Offline
  • Registered: 15-05-2006
  • Posts: 344

Re: Stratiotes aloides

That sort of makes sense. It certainly looks like 0R, 0S and 0T were downloaded at the same time and I can't guarantee that I read and followed the instructions properly. Do you need a copy of my database as well, or would the special upgrade for Rob work anyway. We are on 2S at the moment, looks like I have missed the latest update.

Gordon Barker
Biological Survey Data Manager
National Trust

12 Reply by RobLarge

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

I'm usually pretty careful about reading instructions, and Purgle says she remembers the instruction to run in two batches at some point. However we did have a lot of problems with upgrade kit 3. I recall there being no progress bars and fairly poor error reporting back in those days, so it is quite possible that the update just appeared to hang. If I crashed out of it and then did a visual check for completion, or maybe if this was one of those occasions when we had to manually update the dictionary upgrade record in Recorder, that might explain it.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

13 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

We should  do a special upgrade for you. It will pick up all differences, not just those caused by issues with R, S and T.  It will be slightly easier for me if you can upgrade to  2V before sending me the database.

Mike Weideli

MikeWeideli's Website

14 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

With files of this size, sending on a memory stick is probably best.  Address is 35 Bartlemy Road, Newbury, RG14 6LD

Mike Weideli

MikeWeideli's Website

15 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

The changes were in 0S, which suggest that this didn't run. There was definitely a bug in Upgrade Kit Version 3, which meant that where there was more than one Access file in an update only the first one was processed. however, something else may have gone wrong. Finding out what went wrong and working out a fix is impossible because the problems will have been compounded  by later updates.

Mike Weideli

MikeWeideli's Website

16 Reply by ChrisR

  • ChrisR
  • Administrator
  • Offline
  • Registered: 27-08-2011
  • Posts: 1,131

Re: Stratiotes aloides

If it would help to revive this NHMSYS0000345863 TVK then that is an option. The current name is, technically, ill-formed because it uses an abbreviation of the authority and lacks a publication date so we could bring back the old TVK and then modify one of them to read correctly.

I don't want to complicate matters though - just a thought :)

Chris Raper, Manager of the UK Species Inventory, Angela Marmont Centre for UK Biodiversity,
Natural History Museum, Cromwell Road, London, SW7 5BD.  (tel: 020 7942 5894)
also Tachinid Recording Scheme (http://tachinidae.org.uk/)

17 Reply by RobLarge

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

Thanks Chris, but if it is our dictionary which is out of step, then that is what we must fix. I would be interested to see however, how the various TVKs are associated with the lists after the fix.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

18 Reply by ChrisR

  • ChrisR
  • Administrator
  • Offline
  • Registered: 27-08-2011
  • Posts: 1,131

Re: Stratiotes aloides

Here the only valid TVK for that name is NBNSYS0000002110 because the other one (NHMSYS0000345863) was deleted after it was spotted that it was a duplicate - years ago and before my time. The system should cope with this though because it has a separate table of deleted TVKs. Nowadays we frown on deleting TVKs and I would tend to just edit them to be different and mark one of them ill-formed ... but the trick is for me to spot any mistakes before I send the taxonomy updates to Mike ;)

Chris Raper, Manager of the UK Species Inventory, Angela Marmont Centre for UK Biodiversity,
Natural History Museum, Cromwell Road, London, SW7 5BD.  (tel: 020 7942 5894)
also Tachinid Recording Scheme (http://tachinidae.org.uk/)

19 Reply by RobLarge

  • RobLarge
  • RobLarge
  • Veteran
  • Offline
  • From: Wiltshire
  • Registered: 15-07-2008
  • Posts: 812

Re: Stratiotes aloides

OK Mike, I have finally laid my hands on a memory stick. I will post you a backup file today thanks.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

RobLarge's Website

20 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Stratiotes aloides

Fine Rob. I have completed  the fix for Gordon Barker. There were not many things wrong with the database, but it only takes a few  errors in the relationship to cause problems.

Mike Weideli

MikeWeideli's Website