1 Topic by Matt Smith

  • Matt Smith
  • Veteran
  • Offline
  • Registered: 21-11-2007
  • Posts: 119

Topic: Unverified datasets on the Atlas.

There is a large and developing thread over on the "UK Bees, Wasps and Ants" page at present. One of the group members wondered where an apparent sudden "invasion" of records for the Hairy Footed Flower Bee Anthophora plumipes had come from, as the NBN Atlas map was suddenly covered with records. Digging into the Atlas maps, is is easy to see that the majority of these records come from a very large dataset consisting mostly of unverified records.  See https://www.facebook.com/groups/1590641 … 969198999/

Within the thread are two species distribution maps, one based verified data from iREcord over the last 5 years, and one from the NBN Atlas including the large block of unverified records. Spot the difference.

Discuss.

2 Reply by Darwyn Sumner

  • Darwyn Sumner
  • Darwyn Sumner
  • Veteran
  • Offline
  • From: Dipterists Forum
  • Registered: 10-05-2006
  • Posts: 250

Re: Unverified datasets on the Atlas.

The large block of unverified records originate from the Great British Bee Count, Friends of the Earth datasets.
According to Buglife's page on this "More than 23,000 people around the UK used a free smartphone app to log their sightings of 832,000 bees during the 12 week citizen science survey organised by Buglife, Friends of the Earth and B&Q." - See more at: https://www.buglife.org.uk/great-britis … uhvPd.dpuf
Surveys of this size must surely exceed the capacity of verifiers to verify them, the evidence is slim too.
Datasets that are just plain wrong shouldn't be on the Atlas. This adversely affects any science and subsequent policy as well as acting as a deterrent to schemes uploading their data.

3 Reply by chrisjohnson

  • chrisjohnson
  • Veteran
  • Offline
  • Registered: 18-10-2011
  • Posts: 301

Re: Unverified datasets on the Atlas.

This opens up the whole topic of data verification with respect to large citizen science based projects and the ability of the general public to identify species accurately, with or without an app.
The organisers of such projects, worthy as they are with respect to public engagement and mass data collection, have to take responsibility for the design of their project, vis-a-vis the ability of the public to make an accurate identification, and the verification of the results.
Questionable data from poorly designed projects will harm the reputation of the NBN database, which has always been subject to scepticism about data quality. So I think NBN needs to develop a policy with respect how these datsets are handled.

4 Reply by ChrisR

  • ChrisR
  • Administrator
  • Offline
  • Registered: 27-08-2011
  • Posts: 1,131

Re: Unverified datasets on the Atlas.

Wasn't the FoE Bee Count one of the classic models of how NOT to do a biological survey? Correct me if I'm wrong but I think it involved:
- no verification of records
- no photos attached to records
- no training of surveyors
- a limited choice of species to choose from

Surely this kind of dataset should be switched OFF by default in the mapping of species, otherwise it will render the default maps utterly unreliable. I always thought that the default mapping datasets would come from recording schemes and verified sources, while the unverified data would be available but switched off unless the viewer wanted to see it.

Chris Raper, Manager of the UK Species Inventory, Angela Marmont Centre for UK Biodiversity,
Natural History Museum, Cromwell Road, London, SW7 5BD.  (tel: 020 7942 5894)
also Tachinid Recording Scheme (http://tachinidae.org.uk/)

5 Reply by Mandy Henshall

  • Mandy Henshall
  • Administrator
  • Offline
  • Registered: 31-05-2006
  • Posts: 84

Re: Unverified datasets on the Atlas.

Dear all,

In response to some comments relating to unverified data on the NBN Atlas in this Forum, as well as in the BWARS and NFBR Facebook Groups, please note that any data that was transferred from the NBN Gateway was assumed to be verified as that was the default on the NBN Gateway.

We will be enabling filtering by verified and unverified data on the NBN Atlas in due course, but this is likely to be a few months away yet. In the meantime, you can see the data quality tests that a record has undergone with the results of whether it passed/failed etc.,

We are logging all of this useful feedback in order to enhance the NBN Atlas, so thank you for your comments.

Kind regards,
Mandy

6 Reply by TeresaF

  • TeresaF
  • Veteran
  • Offline
  • From: BTO
  • Registered: 12-05-2008
  • Posts: 171

Re: Unverified datasets on the Atlas.

Hi Mandy,

I see a comment from Richard Comont on the Bees and wasps thread referred to above, that this particular dataset was not publicly visible on the NBN Gateway - presumably because the accuracy issues with this public engagement/app-based dataset have been previously raised and discussed at length (I can remember it coming up at at least one NFBR conference, and probably other meetings too!)

As the problem is partially related to Atlas functionality (no access controls and verification filtering not yet available), perhaps as a special case, removing the Great British Bee Count "stage 1" datasets could be considered, at least until the appropriate filtering is available? The "stage 2" (records verified by photo) ones could stay?
https://registry.nbnatlas.org/datasets# … ritish+Bee


Teresa.

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

7 Reply by Matt Smith

  • Matt Smith
  • Veteran
  • Offline
  • Registered: 21-11-2007
  • Posts: 119

Re: Unverified datasets on the Atlas.

This data was loaded onto the Gateway but with "no public access" and a metadata statement saying "the majority of these records are unverified".   Now the new Atlas has no data access controls and no current way of looking at dataset metadata that I can find, these datasets have "surfaced" and are apparently being given equal weight to datasets with a much more robust verification process behind them.

8 Reply by Jim Bacon (edited by Jim Bacon 25-04-2017 13:55:11)

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: Unverified datasets on the Atlas.

Hi Mandy,

It's not clear if that assumption of all records being verified you mentioned was intentional or accidental. I could easily imagine how I could do the same accidentally. With the knowledge in this thread, though, I'd like to think I would have excluded unverified data until I could handle it rather than the other way round. That would seem to offer the best compromise of service to all parties. If you were to accept Teresa's suggestion to temporarily remove the unverified datasets (are there others?) perhaps you could zip up those data and pop a link on the relevant data resource page so that they are still available to download.

The data quality tests you mention don't obviously help to identify records that are likely to be incorrect as far as I can see. Am I overlooking something there?

Jim Bacon.

9 Reply by charliebarnes

  • charliebarnes
  • Veteran
  • Offline
  • From: Greater Lincolnshire
  • Registered: 07-09-2008
  • Posts: 1,384

Re: Unverified datasets on the Atlas.

Mandy Henshall wrote:

data that was transferred from the NBN Gateway was assumed to be verified as that was the default on the NBN Gateway.

I'm not aware that that was ever the default on the Gateway? (It's how I always wanted the Gateway to 'work' !)

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

charliebarnes's Website

10 Reply by Mandy Henshall

  • Mandy Henshall
  • Administrator
  • Offline
  • Registered: 31-05-2006
  • Posts: 84

Re: Unverified datasets on the Atlas.

Hi all,

We have now removed the three unverified Great British Bee Count datasets from the NBN Atlas, leaving the two which have been verified by photos.

You can see this change on the data partner page https://registry.nbnatlas.org/public/show/dp106

Kind regards,
Mandy

11 Reply by TeresaF

  • TeresaF
  • Veteran
  • Offline
  • From: BTO
  • Registered: 12-05-2008
  • Posts: 171

Re: Unverified datasets on the Atlas.

That's great, many thanks Mandy and the team.

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

12 Reply by Mandy Henshall

  • Mandy Henshall
  • Administrator
  • Offline
  • Registered: 31-05-2006
  • Posts: 84

Re: Unverified datasets on the Atlas.

Thanks Teresa and thanks for your suggestion too.

Regards,
Mandy (and the team!)

13 Reply by dgreves (edited by dgreves 03-05-2017 09:04:34)

  • dgreves
  • Member
  • Offline
  • Registered: 03-11-2016
  • Posts: 35

Re: Unverified datasets on the Atlas.

Equally and oppositely there are many fully documented records languishing in iRecord because some expert organisations have declined to participate, for whatever reason.

Am I right in thinking that there was a verification component to ALA which was not implemented with NBN Altas? Is it ever likely to arrive or would that replicate the work of other existing systems?

14 Reply by charliebarnes

  • charliebarnes
  • Veteran
  • Offline
  • From: Greater Lincolnshire
  • Registered: 07-09-2008
  • Posts: 1,384

Re: Unverified datasets on the Atlas.

dgreves wrote:

Am I right in thinking that there was a verification component to ALA which was not implemented with NBN Altas? Is it ever likely to arrive or would that replicate the work of other existing systems?

I think there is an irecord-style tool built into the software used by the NBN Atlas (see the Atlas of Living Australia: http://sightings.ala.org.au/ but you'll have to log in). From memory it does basically duplicate iRecord, which was why it wasn't/isn't implemented.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

charliebarnes's Website

15 Reply by dgreves

  • dgreves
  • Member
  • Offline
  • Registered: 03-11-2016
  • Posts: 35

Re: Unverified datasets on the Atlas.

I also seem to recall a more iSpot-esque crowdsourced system in a presentation once.

Presumably if funding organisations insisted on data verification as part of any successful project bid then someone would be forced to budget for it. But engagement numbers appear to take precedence. A different discussion perhaps.

My impression of ALA was that it was intended to combine policy, funding and measurable outcomes under a single infrastructure.