1

Topic: TVK errors, reporting and filtering

Hello

I've come across some puzzling behaviours, possibly a problem with our dictionary/previous updates but possibly cuts across a few subjects on the forum, so thought I'd try here first!  I was initially alerted to this after running Record Cleaner on an export containing some newly imported bird records. Record Cleaner gave unrecognized TVK errors on Greenfinch, Siskin, Linnet, Lesser Redpoll, Common (Mealy) Redpoll, Stonechat and Black-Headed Gull records, all of which I believe had been matched to BIRDS, British Ornithologists's Union Cat A, B & C checklist and have different Latin Names in other checklists, e.g. RSPB.

In Recorder, taking Greenfinch as an example, if I run an all taxa report on a specific site there are 560 Greenfinch records (mix of Chloris chloris and Carduelis chloris), whereas if I restrict the report on the same site to Greenfinch from either the Brit Orn A, B & C or RSPB checklists (Choris chloris and Carduelis chloris respectively) I only get 280 records all of which are Carduelis chloris.

Running filters of taxa on the Survey hierarchy also produces inconsistent results for "Greenfinch", "Carduelis chloris" and "Chloris chloris".

In contrast, for Common Lizard, reporting seems to give consistent results for Zootoca vs. Lacerta, though filtering the survey hierarchy on taxon for "Common Lizard", "Zootoca vivipara" and "Lacerta vivipara" all produce different results.

I'm using Recorder 6.22 with dictionary 28.

Any suggestions as to what is going on appreciated. And regarding filtering of the survey hierarchy on taxa, is that just the way it works (i.e. have to mindful that it will be sensitive to use of different names) or do we have a problem with that too.

Best wishes,
Andy.

2

Re: TVK errors, reporting and filtering

Could be that R6 is now ahead of the Record Cleaner and the Gateway. R6 is reflecting what is in the NHM dictionary as far as  the Greenfinch is concerned.   I  don't know enough about the taxonomy to know what is correct, but Chloris chloris and Carduelis chloris are considered as two separate species, with Chloris chloris not having any link to the common name 'Greenfinch'. 

I need to investigate the lizards a bit further.

Mike Weideli

3

Re: TVK errors, reporting and filtering

AFAIK Chloris chloris should be synonymized with Carduelis chloris (I think Chloris chloris is the newer name). Chris may need to wade in ?

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

4

Re: TVK errors, reporting and filtering

Thanks for the responses. Yes, as far as I can tell they are the same, with Chloris newer - can see from the metadata this has been edited recently on the Brit Orn Cat A,B,C list. Presumably similar issues with name changes for the other birds mentioned, though the Redpolls might be a special case as a spilt with respect to the RSPB checklist.

Is the behaviour of the filtering I mentioned unexpected? Just curious as to whether I should stop using this feature as I have done heretofore.

Best,
Andy

5

Re: TVK errors, reporting and filtering

I always assumed that the filtering just worked on what's displayed on the screen regardless of synonymy etc.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

6

Re: TVK errors, reporting and filtering

Just had a look at our data and I get 38 records looking at greenfinch in BIRDS and 143 with R3.3. It looks to me as if there is no synonymy between the two lists, although some other lists do synonomise to R3.3 where appropriate, e.g. VASCULAR PLANTS. Some might link through an intermediary table (Common lizard?) . This does assumes that my view of the taxon tables is close to reality.

Gordon Barker
Biological Survey Data Manager
National Trust

7

Re: TVK errors, reporting and filtering

The Greenfinch looks like a dictionary problem as Carduelis chloris is not  being treated  a synonm of  Chloris chloris. 

Something I have discovered looking at the Common Lizard, suggest there may be an  issue about how Recorder is  finding related taxa, but a lot more investigation is required and I may be wrong.

Mike Weideli

8

Re: TVK errors, reporting and filtering

Investigation shows a problem where some taxa will not be picked up if they were entered from a Designation List. Entering or matching agaisnt these lists  isn't possible now, unless matching is done outside Recorder,  but it was for a while. As far as I can see there shouldn't be any issue with the common lizard unless the Taxon_List_Item_Key came from one of these designation lists.  Apart from taxa on the designation lists there are 45 other taxa affected.  Nine  taxa ( 4 species) on the Diptera check list and the rest on the Environment Agency Check List. Changing R6 to deal with this problem is going to take some detailed investigation, because making what seems the obvious change may have a knock on effect elsewhere. As an interim measure it might be possible to make some adjustments via the dictionary which will get around the problem and I will look into this.

Apart from these exceptions I  think that if the links in the dictionary are correct, then R6 will find all related taxa. 

Andy - it may be worth running some further tests on your system to see if the problem with the lizard is down to entries coming from designation lists and also to make sure that your dictionary hasn't got out of step at some point.

The problem with the record cleaner is probably down to the fact that R6 is in line with the latest NHM dictionary and ahead of the gateway. This has proved the case with some other recent queries.

Mike Weideli

9 (edited by andy_f 17-04-2014 11:02:56)

Re: TVK errors, reporting and filtering

Hi Mike, please could you advise on further tests?  I'm struggling to link back records to which checklist they were matched to - TVK not available in reports, and if I try to select Taxon Checklist as a report field I get a crash with EOleException, String or binary data would be truncated - presumably the name of one or more checklist is too long for how reporting is setup?
As TVK etc are not in the TAXON_OCCURRENCE table I'm not sure how to relate a specific occurrence to a checklist directly from the database.

The behaviour on filtering is:
Filtering on "Zootoca vivipara" produces the fewest results - all Z. vivipara
Filtering on "Common Lizard" produces an intermediate number of results - mixture of L. and Z. vivipara
Filtering on "Lacerta vivipara" produces the most results - mixture of L. and Z. vivipara

Picking another species with a change in latin name, filtering for water vole gives me these results:
Filtering on "Water vole" - fewest results - all A. terrestris
Filtering on "Arvicola amphibius" - intermediate number of results - all A. amphibius
Filtering on "Arvicola terrestris" - most results - mix A. amphibius and A. terrestris

Cheers,
Andy.

10

Re: TVK errors, reporting and filtering

P.S. as regards the unrecognized TVK errors with the bird data mentioned in my original post - I was about to send an export including these to our LRC. Can I take it that these are fine and will be sorted in time, and that it's OK to go ahead and pass on the export?

Many thanks,
Andy.

11

Re: TVK errors, reporting and filtering

Andy

Just one thing we should check before doing anything else. In Tools Options do you have Use Translation to Preferred Term ticked.

A dictionary upgarde will correct any problems with the birds, but not sure when Chris will be able to do this.  As long as you have extracted all the required records there shouldn't be any problem.

What were you using to try and get the Survey name. Report wizard seems to have correct length for the field name.

Mike Weideli

12

Re: TVK errors, reporting and filtering

Hi Mike

Yes, the Use Translation... box is ticked.

Thanks for confirmation about the bird TVK issues.

I was getting the crashes from the report wizard if I selected "Taxon Checklist" under Taxon on the Select Attributes screen of the wizard - all fine until the report actually tries to run, then crashes out with the above error. If I continued the program and went backwards through the wizard and unticked Taxon Checklist then the report completed without error.  However, further investigation has shown that I only get this crash if I'm trying to report on all taxa (for a polygon or Recorder location) - if I specify a Checklist or species on the Taxa Selection form of the wizard I don't get the crash.

The crash seems to occur on most specified polygons/locations, however I tried using a location with only two plant records as the location criterion, and that report didn't crash. So, I'm guessing it is something to do with our data, possibly taxa matched against checklists no longer in use? 

So, using a report for all species on the prefered Reptilia checklist, all the common lizard records have been matched against either REPTILIA Nat Hist Mus or Recorder 3.3 (1998) checklists. However, this wouldn't tell me if there are other common lizard records matched against other checklist if the report isn't actually picking them up... (hence desire to do all taxa reports with "Taxon Checklist" field).

Cheers,
Andy.

13

Re: TVK errors, reporting and filtering

This could be something we have missed in the latest upgrade, but I can't duplicate the problem. From a dictionary point of view it looks like all teh links are there to report on the Common Lizard irrespective of which list or common/scientific name you use.   If I send you an xml report to check out a few things will you be able to run it ?

Mike Weideli

14

Re: TVK errors, reporting and filtering

Sure, thanks Mike.

Best,
Andy.

15

Re: TVK errors, reporting and filtering

The attached report which runs from a menu named 'Test Report'  should return 19 rows, which  are all possible entries for the common lizard under various names.  Please let me know the results of running this on your system. To install just copy the file into the reports folder of R6.

Post's attachments

Lizard_Dictionary_Check1.xml 832 b, 3 downloads since 2014-04-22 

You don't have the permssions to download the attachments of this post.
Mike Weideli

16

Re: TVK errors, reporting and filtering

Hi Mike - the report returns 19 rows. Just out of interest, is the Recommended Taxon List Item Key the unifying reference for the various names of a species in the dictionary? Was wondering why none of the keys I've previously seen were common to the various names, but assumed there must be a common key somewhere.

Thanks,
Andy.

17

Re: TVK errors, reporting and filtering

P.S. just in case the report itself is of any use...

Post's attachments

LizardReport.xlsx 7.83 kb, 2 downloads since 2014-04-24 

You don't have the permssions to download the attachments of this post.

18

Re: TVK errors, reporting and filtering

Thanks. This shows your dictionary is fine, but fails to explain why you are not getting the correct results when you report on these names.  If you start with any of the Taxon_List_Item_Keys you should get everything related to them.  I have run some more test, but can't at the moment detect any issues, so will need to investigate further. The Recommended_Taxon_List_Item key is unifying reference.

Mike Weideli

19

Re: TVK errors, reporting and filtering

Thanks Mike. Just to clarify, as per my original post, from the small number of test reports I've done the results seem consistent for Common Lizard - it is the filtering on Taxon of the Survey Hierarchy that is inconsistent according to the name used to filter (and others like Water Vole).  This may be irrelevant, but thought I'd just highlight in case it means something different is causing a problem.

Best,
Andy.

20

Re: TVK errors, reporting and filtering

Interesting, I had never even noticed that you could filter like that. Learn something new every day. Thanks (even if it is inconsistent.)

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

21

Re: TVK errors, reporting and filtering

It could well be that we have missed as far as this filter is concerned. Will check it out.

Mike Weideli

22

Re: TVK errors, reporting and filtering

I've been doing a bit more testing to clarify what is going on with the crash I'm getting when I running a report as described above. From what I can tell by applying various filters to my data, when using the Reporting Wizard, if I select 'Taxon Checklist' as an Attribute for my report the report crashes if it comes across any taxon occurrences matched to a dictionary with a title more than 100 characters long. For me these are:

NEUROPTERA, MEGALOPTERA RAPHIDIOPTERA & MECOPTERA Checklist of British lacewings and their allies (Neuroptera, Megaloptera, Raphidioptera and Mecoptera)
FEP Guidance 007: Plant & Animal Species in the Farm Environment Plan (FEP). Table 2: Species to be recorded in Part 2 of the FEP
ISOPODA Checklist of woodlice (Isopoda: Oniscidea) and water-slaters (Isopoda: Asellidae) occurring in the British Isles
DIPTERA Checklist of Insects of the British Isles (New Series) Part 1: Diptera (Chandler, 1998 and updates)
CRUSTACEA Fauna Europaea checklist of freshwater and terrestrial Crustacea of Britain and Northern Ireland
FUNGI British Mycological Society Preferred List of Fungal Names - Includes Lichen Names (non preferred)
LEPIDOPTERA Checklist of Lepidoptera Recorded from the British Isles (Second Edition; Bradley, 2000)
THYSANURA Checklist of British Thysanura (silverfish and firebrats) and Archaeognatha (bristletails)

(error: EOleException, String or a binary data would be truncated)


Apologies if I should have started a new thread for this, thought I'd post here as I'd mentioned it previously in this thread.

Cheers,
Andy.

23 (edited by robtaylor 02-04-2015 10:08:43)

Re: TVK errors, reporting and filtering

Hi,
We are having some problems using the export wizard add-in to do an NBN exchange format report, we are getting an unhandled 'String or binary data would be truncated.' exception which causes exit of Recorder6, I have attached an example of the error output.
This issue is happening with various different taxa one of which is Bryophites, to narrow it down somewhat we have repeated the error isolating the sub-class Bryopsida as being the cause.
Could this be a dictionary problem, i.e. Bryopsida contains a variant with a pretty long Latin name?
Or could it be a field in our data which is too long?
Thanks
Rob

Post's attachments

Error-73569038299105.txt 36.16 kb, 5 downloads since 2015-04-02 

You don't have the permssions to download the attachments of this post.

24

Re: TVK errors, reporting and filtering

Do you know which version of the addin you are using ?  Don't think it is dictionary problem, but more likely  one of the other fields.

Mike Weideli

25

Re: TVK errors, reporting and filtering

Thanks
The export filter is v6.22.