1

Topic: Indicating verification status

As we make moves to improve the quality of the data we hold by using standardised rules of the NBN Record Cleaner plus the input of Vice County Recorders and other experts, I'm interested in the different approaches taken to indicate verification status of records in Recorder 6.

The way I've seen it done uses the batch update 'Update determination based on spreadsheet', which creates a new 'preferred' determination.

One school of thought is to use this to create an entirely new determination, for example:
'NBN Record Cleaner' as the determiner, ‘Considered correct’ / ‘Considered incorrect’ depending on whether a record passed or failed the rules, date of determination being the date the tool was run and perhaps the rule passed / failed in the Det comments
and a similar approach to reflect verification feedback from County Recorders, etc.

This doesn't sit quite right to me as you then effectively lose the original determination, and what is happening with this process is not determination, but verification.

Another approach could be to create a new determination, but leave the Determiner as is and just change the determination type to 'Considered Correct' (or 'Incorrect') and add a comment to the determination stating 'Passed/Failed Record Cleaner <date>'.
again with a similar approach for feedback from County Recorders, etc. unless re-determining an incorrect record.

How do other users approach this?

Linds

Lindsay Bamforth
Fife Nature Records Centre,
Information Officer

2

Re: Indicating verification status

We had the same concerns over adding a new determination as part of verification - being named as determiner on a record they hadn't personally identified understandably wasn't popular as an idea with our verifiers. We implemented the last approach with the help of Mike last year but amend the existing determination rather than create a new one.

We use a batch update to change the determination type to "considered correct" but retaining the original determiner and determination date, and appends a suitable determination comment to any existing determination comment, e.g. [Verified Cumbria moth referee NAME DATE]. There is a 100 character limit on the determination comment. This is done using a spreadsheet containing three fields - the taxon observation key of each record, the determination type and the verifier's comments. Any records that are not "considered correct" in the determination type field are ignored so we can go and add new determinations to these manually. I'm really happy with the results - it is much easier to work out the verification status of our recent records now and over time we hope to get to the stage that all records in the database will have such a metadata trail for verification.

We'd be happy to share the batch update but it would need Mike's permission.

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

3

Re: Indicating verification status

No problem in sharing as far as I am concerned. It is down to everyone to decide what they wish to do and Recorder is capable of dealing with most ways of approaching this. The basic verification system can be adapted to do whatever is required


1. The concept behind R6 is that a  Determiner is a person who is making the decision about the taxon allocated to the record. There can be multiple determination for a record, only one of which can be the preferred determination. When a record is validated the person becomes the determiner, because they are saying that from the evidence they have, that the determination is correct or otherwise. They may in some circumstances actually change the determination. Validation is just a special form of determination indicated by the determination type and the determiner role. The original determiner is not lost., but retained on the non-preferred determination.   This approach adds a new Determination in every case and is the one used in the Validation System which is generally available.

2.  A slightly different view is that when a record is verified, the verification is rubber stamping or otherwise what is already recorded and creating a new Determination is unnecessary.  There are a few different approaches used here :-   


a. If the person doing the validation is happy with the record then the determination type is changed, but not the determiner. No other change is made to the record. Only if the person doing the validation disagrees is a change made to the Determination, Determiner and Determination Type. This approach can only apply in circumstances where the system is the custodian of the record, otherwise a new determination has to be created.

b. The disadvantage of approach a.  is that there is no track of who actually validated the record. So a variation is that the Determiner is changed if the Recorder is currently the Determiner. If the Recorder is not the Determiner then  this is treated as a new determination. 


c.  A further variation as used by Cumbria where the comment field is used to track the person doing the validation where there is no disagreement.

It has been suggested that the verifier is added as a separate field to the Determination, but  there is some opposition to this on the basis that the R6 data model model already adequately deals with the situation. 

Last point is that the 100 character import  limit on the Batch Update will be changed to a 255 character limit in 6.24.

Mike Weideli

4

Re: Indicating verification status

Thank you both for all of that useful information.  Really helpful to hear others' thoughts and approaches used.

I like the approach referred to by Teresa to keep the original determination information where the verification agrees with it and append the verification comment.  Particularly from a point of view of trying to include maximum information within an export type layout in the absence of separate verification fields, where not everything that's in Recorder is visible.

Where custodianship of a record is not held, is there a way to 'pull through' the information from the original determination to create the same effect but within the necessary new determination?  Or could the batch update be amended to always retain the original determination, but also create a duplicate as a new determination, in which the determination type is amended and verification comment appended?

Have you ever had the situation of records being reverified?  We are thinking about running records through the NBN Record Cleaner en masse as an 'initial sweep' and relatively quick way of achieving improvement in quality, but with a view to gradually enlisting the help of County Recorders and other human expert verifiers to take a look.  The reasoning being that they can use local knowledge, flexibility, etc. that standardised national rules cannot.  Sometimes the verifier may only wish to or have time to look at the 'failed' records that require further investigation, but others like to see the full dataset.  A record could therefore end up being verified by both the Record Cleaner and the expert.  Presumably running the batch update again would append any further verification onto the existing extended comment (character limit permitting)?  Though perhaps once you have the human verification, the NBN Record Cleaner comment may be surplus to requirement and it would be more desirable to be able to just append the comment to the original determination comment without the 'intermediate' stage?

Linds

Lindsay Bamforth
Fife Nature Records Centre,
Information Officer

5

Re: Indicating verification status

In cases where you are not the custodian the Batch Update can easily create a new preferred determination based on the existing determination - this is what the standard update does for every record.

As far as reverification  is concerned you could use  different determination types. Say one for  NBN Record Cleaner and another for County Expert etc. You can append to the comment each time, but the determination type/determiner  will only be for the   latest one recorded.  (This is one reason why the R6 standard  creates new determinations each time as this tracks the full history).

Mike Weideli

6

Re: Indicating verification status

MikeWeideli wrote:

In cases where you are not the custodian the Batch Update can easily create a new preferred determination based on the existing determination - this is what the standard update does for every record.

I thought the standard update created an entirely new determination based on what you enter in an Excel file rather than automatically using information from the existing determination?  Or have I misunderstood?

What I'm thinking of is a way to achieve a result similar to what Teresa describes but in a new determination (i.e. same determination date and determiner as the original determination, but verification comment appended to original determination comment).  Would we do this by entering this original information into the Excel file to be used to create the new determination and concatenating the original comment with any verification comment we want to append into the field that will create the new determination comment?  Or is there a better way?

Thanks for your help,
Linds

Lindsay Bamforth
Fife Nature Records Centre,
Information Officer

7

Re: Indicating verification status

It is a new preferred determination which is created by the standard Batch update.  It will copy over any information not included in the Excel file, so for example if a new species isn't provided then the previous one is copied over.  I think you should  be able to use Teresa's Batch update without any problems and that this will do what you need. I don't think there is anything  in there specific to Teresa's version of R6, but if there is it can easily be changed.

Mike Weideli

8

Re: Indicating verification status

Great - thanks for the clarification Mike.

Teresa - please could we take up your offer of sharing the batch update you described?

Many thanks both,
Linds

Lindsay Bamforth
Fife Nature Records Centre,
Information Officer

9

Re: Indicating verification status

I have attempted through the R6 Steering Group to get the validator, reviewer, verifier etc. including as a separate  field in Recorder. This feature has been requested in different forms by a number of users and I was attempting to get something which met all the requirements.. Unfortunately, the proposal has been rejected so I can not proceed.

I am exploring another solution which will  not change the data model.

Mike Weideli