1 Topic by BDeed

  • BDeed
  • Veteran
  • Offline
  • From: Bolton
  • Registered: 01-03-2011
  • Posts: 447

Topic: CEH species rules

I previously posted this in the Record Cleaner thread but there doesn't seem to be much of any activity over there and as this affects Indicia recording i thought i'd repost here on the offchance someone could help or at least point me in the right direction. Tenuous link complete...


We are in the process of setting up an Indicia based recording system and have a limited use trial going on. We are using the CEH record cleaner(?) rules to provide automated notifications of records that are out of the ordinary.

I've just had a query as to why Pink-footed Goose has been flagged up as in flight out of season.

The following message was attached to your record of Pink-footed Goose at SD334014 on 06/12/2015 when it was checked using the NBN Record Cleaner...  ...Date is outside usual flight period of Pink-footed Goose.

PFG is a very significant species in Merseyside & West Lancs. at this time of year! Is there any way this notification can be amended?

Natural History & Biodiversity Data Enthusiast

2 Reply by Jim Bacon (edited by Jim Bacon 16-12-2015 11:47:24)

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: CEH species rules

Hi

I think your original post was in the correct place. The rule needs updating at source which is the NBN. Graham French has dealt with issues like this in the past. (http://www.nbn.org.uk/The-NBN/The-Staff … ccess.aspx)

I'm not sure how Indicia gets/maintains its copy of the rules but modifying locally is unlikely to be a good option.

In answer to the question of why this has happened, to quote http://forums.nbn.org.uk/viewtopic.php?id=4399,

Flagging up of common species may be the result of missing 10km squares within the species range in the rule file. This is likely to be due to under-recorded squares where records are not in the national scheme's database used to generate the rules.

I'm only quoting so don't blame me if that seems unlikely! Graham goes on to say

If you send me (Graham French) the suggested corrections I will have a look to see if this is the case or if this is a problem with the rules themselves.

Jim Bacon.

3 Reply by BDeed

  • BDeed
  • Veteran
  • Offline
  • From: Bolton
  • Registered: 01-03-2011
  • Posts: 447

Re: CEH species rules

Cheers Jim,

I'll contact Graham directly when i get a sec.

Natural History & Biodiversity Data Enthusiast

4 Reply by Graham French

  • Graham French
  • Administrator
  • Offline
  • Registered: 23-05-2006
  • Posts: 430

Re: CEH species rules

Hello Ben

The issue seems to be that the rule specifying the seasonal period for Pink Footed Goose is a reverse rule so that records will be
flagged up if they occur between the start and end dates and not outside them. This is so that the rule can handle wintering species
whose date range covers dates across years (eg September to June). The rule is

[Metadata]
TestType=PeriodWithinYear
Group=Birds Flight Period
ShortName=Pink-footed Goose
Description=Checks date is within most regular occurrence period for Pink-footed Goose
ErrorMsg=Date is outside usual flight period of Pink-footed Goose
Tvk=NHMSYS0000530150
DataFieldName=
StartDate=0602
EndDate=0914
ReverseRule=True
LastChanged=20110826
[EndMetadata]

So for Pink Footed Goose this is between 2nd June and 14th September. Records outside these dates, like 06/12/2015 should pass the rule. The NBN Record Cleaner applies this reverse rule but presumably Indicia does not so that records after 14th September are incorrectly flagged up. If this is the case the issue isn't with the rule but how it is handled by Indicia.

Best wishes

Graham

Graham French's Website

5 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: CEH species rules

Thanks Graham, that is really helpful.

I have identified the rule in the Indicia database verification_rules table (title='Pink-footed Goose' AND test_type='PeriodWithinYear'). The reverse_rule field is set to FALSE which is incorrect.

Moreover, there are no bird rules in our database with reverse_rule = TRUE whereas I count 39 such in the BTO Bird rules/flightperiod files.

This suggests that the source of the error is on import. A little look shows that there was indeed a bug, now fixed, that would have led to the import failing in this way (https://github.com/Indicia-Team/warehou … 6f851d3072). While the bug was fixed, it does not appear that the rules have been re-imported to correct the database with the possible exception of the moth rules.

I will just confer with colleagues on how to proceed. Thanks BDeed for drawing this to our attention in the first place.

Jim Bacon.

6 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: CEH species rules

Hi

I have just run the import of the record cleaner rule sets for birds and am now seeing 39 reverse rules as expected. Hopefully that should fix the problem you have encountered, BDeed. Please test it and let us know.

At the same time Indicia gained geographical rules for 352 bird species. If you record outside the expected range for a species you will now receive warnings related to this.

I will continue with updates to any other rule set which may have errors or changes.

Jim Bacon.

7 Reply by BDeed

  • BDeed
  • Veteran
  • Offline
  • From: Bolton
  • Registered: 01-03-2011
  • Posts: 447

Re: CEH species rules

Thank you all for getting this sorted. I've entered two records. One for June and one for Jan and only the June record received a notification so appears to be spot on now.

Thanks again,
Ben

Natural History & Biodiversity Data Enthusiast