1 Topic by robtaylor

  • robtaylor
  • Member
  • Offline
  • Registered: 13-11-2014
  • Posts: 20

Topic: Output differences when reporting taxa by class and species

Hello all,
I wonder if anyone else is experiencing the same as us...

We are experiencing a slight problem in using the R6 reporting wizard, we stumbled on this problem accidently and believe that it would be too much of a coincidence for it to be limited to the 1 example we found.

The problem:
When using the report wizard for specific species: 'Red-Eared Terrapin', we got 21 entries, when reporting all reptilia, we only get 8 red-eared terrapin records.

It seems that there is a sub-species which shows up on the search for 'red-eared terrapin' but not in the reptilia report.

We are using version Recorder 6 v6.22.
the main species is Trachemys scripta (TVK NHMSYS0000080227)
the sub species is Trachemys scripta subsp. elegans (TVK NHMSYS0000375431)

Other interesting anomalies:
-the sub-species doesn't appear in the hierarchy for Replilia but is on NBN gateway.
-the report shows the class, order and family of all 21 records as 'Animalia, which is obviously wrong but doesn't prevent the records getting to the Reptilia report.
-there are 3 list_item_keys (NBNSYS0000093663, NHMSYS0000332351, NBNSYS0000093662), the first relating to the sub-species.

Any help would be appreciated

2 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

In the Dictionary (Organism table) the parent of the genus Trachemys is animalia. This will stop any reports down the hierarchy from Reptilia from working. I expect this was originally because it wasn't considered a British species . This will be easy to change and if you require an immediate fix this could be provided.

However, what isn't clear is why you get even 8 records when reporting on Reptilia as I would expect none, which is what I am getting with some test data.  Please could you advise what criteria you are using when getting 8 records out of the report.

Trachemys scripta subsp. elegans is on the list of additional names, which is why it doesn't appear in the hierarchy on the  Reptilia preferred list.

Also please could you let me know which dictionary version you are on. This  may have an impact on further investigation.

Mike Weideli

MikeWeideli's Website

3 Reply by robtaylor

  • robtaylor
  • Member
  • Offline
  • Registered: 13-11-2014
  • Posts: 20

Re: Output differences when reporting taxa by class and species

Hi Mike,
Thanks for the swift response.

Thanks for the offer of immediate fix, however I am unsure whether we need/want this yet, I will need to discuss further with colleagues.

The criteria we are using to pull the 8 records: in taxa selection, select REPTILIA list tick all taxa from list, using full report template. When the report shows, filtering via the text red-eared, this brings up the 8 little blighters.

The criteria where we get the 21 records is the same however we use the search box under REPTILIA list to search for red-eared, which brings up the Trachemys Scripta, selecting this with full report produces 21 records.

Our dictionary was #23, I did think it could be a dictionary out of date issue, so I applied all the newer dictionary updates. The dictionary updates seemed to apply correctly, however caused a further reporting problem (all individual taxon searches pulled nothing in reports). I therefore had to restore the database to before dictionary updates, the data now behaves as previously described. Just checked and our dictionary is showing as 2B now.

I am going to have another go at updating, this time checking in-between batches for the same report problems, I did fully follow all install instructions to the letter, i.e. full database backup, installed dictionary update v4, unzipped each batch of updates to correct script folder, run executable, rebuild indexes in order group, name, synonym, designation after each batch.

Let me know if you need any further info
Thanks for your help
Rob

4 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

I think it probable that when you ran the report with 'Select all taxa from list' , 'Expand Taxonomic Group ' wasn't ticked. This would mean the the sub-species wouldn't be picked up, because it isn't on the Reptiles list, but in the List of additional name.

If this is the case then I think the whole problem is because the Genus points to Animalia and not to the family. Chris at the NHM has now fixed the master list so the change will be in the next dictionary upgrade (000002W), but I could provide this immediately if it helps.


As far as the dictionary upgrade is concerned.  Once you are beyond dictionary 0000001N I can't think of any reason for species going missing from reports other than one of the index tables not building correctly. The index tables are cleared down as part of the dictionary processing, but all the others are just updated.

If you running more than one set of updates you only need to rebuild the indexes once after the last one.

Mike Weideli

MikeWeideli's Website

5 Reply by robtaylor

  • robtaylor
  • Member
  • Offline
  • Registered: 13-11-2014
  • Posts: 20

Re: Output differences when reporting taxa by class and species

Thanks Mike,
The 'Expand Taxonomic Group' check box was definitely ticked on the report wizard, have just re-checked to make sure.
I think we are on the right track, though.
-All of the red-eared species come out of the report with 'Animalia' as class, order and family, but only the main species appears in the taxon hierarchy under reptiles/turtles/emydidae
-The taxon hierarchy doesn't show Trachemys Scripta as having a sub-species so the 'Elegans' sub-species is not contained within the Reptile class, nor any other class as far as I can tell.
I guess this is why it doesn't appear in the list.
I will do the dictionary update.
If you have a copy of the 000002W update, this will be useful for us to test.
I will let you know how it goes.
Rob
Thanks

6 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

Many thanks.

I am comfortable that this problem is just down to the dictionary issue and that we have a fix. I will investigate why expanding the taxon group isn't picking up the sub species as I would expect, but  this is a different issue.

I will work out what is needed  2W and post it later.

Mike Weideli

MikeWeideli's Website

7 Reply by robtaylor

  • robtaylor
  • Member
  • Offline
  • Registered: 13-11-2014
  • Posts: 20

Re: Output differences when reporting taxa by class and species

Mike,
I re-applied all the dictionary updates, then rebuilt the indexes and the same problem as before is occurring, there is nothing returned from an individual species report (for any species).
I have now restored the database.
I take it from what you mentioned in one of your previous posts in this thread, I can put all the available updates into the script folder and run the update exe? the executable will then apply the updates in order, I can then just build each index at the end of this process?
Thanks
Rob

8 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

You can put all available scripts in the scripts folder and do just one run with the index rebuilds at the end, but I think we should find out what is happening with your reports before proceeding further. What you are experiencing  isn't  normal.

I didn't notice before but the correct order for Index Rebuild is Name,Group,Synonym then Designation which is the order they appear in  the Database tools.  Group relies on Name being populated.  If I am responsible for giving you the wrong order at some point in the past please accept my apologies.

Mike Weideli

MikeWeideli's Website

9 Reply by robtaylor

  • robtaylor
  • Member
  • Offline
  • Registered: 13-11-2014
  • Posts: 20

Re: Output differences when reporting taxa by class and species

I thought it was odd that the order in the menu was different to that on the update page:

"Rebuild indexes

Once the upgrade has completed go to Tools - Database Tools in Recorder 6 and rebuild the taxon group, taxon name, taxon synonym and taxon designation indexes. Note that rebuilding the taxon designation index must be done last as it uses the other indexes. Depending on the power of your computer(s) and the speed of your network, where applicable, each rebuild may take 20 minutes to run. On slower systems they may take somewhat longer.
"
So you don't think that updating the dictionary will solve our problem?
Do you think I should re-do the upgrades, this time re-building in the correct order?
Can I provide any more information you need to be able to help?
Thanks
Rob

10 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

After version 6.18 the order changed and it became important that Name was built before Group. Instructions issued prior to that became  out of date. The correct order no matter which version of R6 you are on is the order they appear in Database tools.

I think you should be able to do just one update. Make sure all the script files  you need are in the Scripts folder, including 2W which is attached. It will take a while to run, but you should  get progress bars.  Run the index rebuilds in the order they appear in database  tools and you should  have a up to date dictionary with the  Trachemys problem fixed

Post's attachments

0000002W.sql 344 b, 2 downloads since 2015-02-18 

You don't have the permssions to download the attachments of this post.
Mike Weideli

MikeWeideli's Website

11 Reply by robtaylor

  • robtaylor
  • Member
  • Offline
  • Registered: 13-11-2014
  • Posts: 20

Re: Output differences when reporting taxa by class and species

The dictionary update and index build now ok, thanks.
The 'Animalia' for class order and family now ok thanks to update 2W.
Problem persists where sub-species not showing in Reptilia, but shows search on all hierarchical parents in-between.

12 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

There is either a bug or at least an anomaly in the way reporting is working when 'All Taxa From List' is selected, it is just reporting on the taxon on the actual list selected even if 'Expand Taxonomic Hierarchy' is selected.  This means that you when you are in NHM Reptiles list and select 'All Taxon from list' then you don't get out the sub -species as it isn't in the list, but only in the 'List of Additional Names'.  However, if instead of selecting 'All  Taxa on  List' you use  'Find Taxon'  and select Reptilia, then you will get all the reptiles including the sub-genus.

Mike Weideli

MikeWeideli's Website

13 Reply by PhilippaTomlinson

  • PhilippaTomlinson
  • PhilippaTomlinson
  • Veteran
  • Offline
  • From: Isle of Man
  • Registered: 18-11-2010
  • Posts: 54

Re: Output differences when reporting taxa by class and species

I'm so pleased to be able to dowload the most recent batch of dictionary updates, many thanks!
Please note that the instructions on the Dictionary Download page http://jncc.defra.gov.uk/page-4615 re the order that you should run the re-build of indexes is wrong. Although the order in the menu in Recorder is correct I nearly did it the wrong way....
Could this be corrected on the webpage?
many thanks
Philippa

Dr Philippa Tomlinson
Biological Records Manager
Manx Biological Recording Partnership

PhilippaTomlinson's Website

14 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,645

Re: Output differences when reporting taxa by class and species

Thanks. will get JNCC to make the change.

Mike Weideli

MikeWeideli's Website