1 (edited by ALERC_NC 03-05-2018 15:51:11)

Topic: GBIF

Apologies if this is widely known, or mentioned elsewhere, but does or will the Atlas accept records from GBIF?  I am aware that NBNT are the official UK node for GBIF, and supply records from the UK to GBIF, but does data come back the other way?

Thanks,

Tom Hunt - Association of Local Environmental Record Centres, National Coordinator

2 (edited by Darwyn Sumner 07-05-2018 16:54:34)

Re: GBIF

I don't think it works that way Tom. Once a node has uploaded a dataset to GBIF, that's the end of it unless it is resubmitted.
Though GBIF is another GBG (Global Biodiversity Gateway) like our NBN Atlas, it covers a wider geographical area. with inputs differing from the Atlas. Outside the UK in the main they seem to be data from projects organised by a countries institute (see list of nodes), many of them being museums and/or academe.
Our NBN is unique, no other country has anything similar to its scope and infrastructure which reflects two centuries of activities by naturalist recorders.
I know of European workers who visit the UK, they've all used our NSSs to submit species occurrences.
If you are investigating datasets which are on GBIF and may contain UK records then I suggest you focus on museums listed on GBIF's site, I recollect spotting a Scottish museum's plant dataset that was sent directly to GBIF
I wish there were NBN equivalents in other countries

3

Re: GBIF

Hi Tom,
Currently we don't receive any datasets from GBIF, so, no, we only send data to GBIF. If there are UK-based datasets on GBIF that the community would like to have on the NBN Atlas I don't see any reason why we couldn't pull the data into the Atlas, as long as the data owner was happy with it.

Best wishes, Sophie

Sophia Ratcliffe
NBN Atlas Data Manager

4

Re: GBIF

The GBIF analyse tool has a handy 'repatriated' data button - from memory I think this filters datasets that cover any geographic area other than that of the contributing node.

Data owner permission would not be needed to transfer to the NBN as all GBIF data is covered by the same licences that the NBN offers.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

5

Re: GBIF

Thanks for the responses.

There may or may not be that much UK data on GBIF from foreign sources.  It could be an interesting thing to find out if anyone had time to look.

What might have a more significant impact is iNaturalist https://www.inaturalist.org/ when it used by UK recorders.  As a global online recording system, it sends its data to GBIF.  But could an agreement be reached between NBNT and iNaturalist that sees data come to the Atlas?  Of course this may already be in discussion, in which case great.

Regards,

Tom Hunt - Association of Local Environmental Record Centres, National Coordinator

6

Re: GBIF

1,229,333 records - https://www.gbif.org/occurrence/search? … advanced=1

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

7

Re: GBIF

Thanks.

77 000 on iNaturalist for the UK.  I don't know how many of them have reached GBIF, but whatever the figure, it doesn't look there is that much UK data on either platform yet.  It will be worth keeping an eye on though and maybe forming some kind of arrangement in the future.

Regards,

Tom Hunt - Association of Local Environmental Record Centres, National Coordinator

8

Re: GBIF

The problem with iNaturalist is that is has a "crowd-sourced" ID system where the preferred ID depends on the number of "votes" it gets.  It works in a similar way to iSpot - in neither system is the data verified in the way that it is on iRecord. 

I believe there is some initial testing of a possible link from iNaturalist to iRecord ongoing, which means that UK records may end up on the NBN that way at some point.  For both iNaturalist and iSpot data, I'd have to say that any data making its way directly from either system onto the NBN would have to be marked as "Unconfirmed".

9

Re: GBIF

A link between iNaturalist, iSpot or any online recording system for that matter, to iRecord sounds great.

Thanks,

Tom Hunt - Association of Local Environmental Record Centres, National Coordinator

10

Re: GBIF

I don't suppose there are many UK Recording Schemes wishing to obtain a good distribution picture for Europe, I'm sure to be in a minority (perhaps birds & plants though.) See http://www.micropezids.myspecies.info/node/353 where I've made a start on Country checklists.
Dots on GBIF Europe maps is my aim.
To that end I've been looking at methods to achieve that. Unfortunately the international method of iNaturalist only accepts images (that's the evidence-base for their verification system) and most European countries don't have nodes (see https://www.gbif.org/the-gbif-network/europe), those that do, you'd have to speak the language.
I have many records from published papers, foreign expeditions, images posted on sites such as Flickr etc. and no obvious method to get them onto GBIF.
Accordingly I applied to GBIF to become an "endorsed organization", surprisingly successfully thanks to NBN.
I'm now puzzled as to know how to submit data to them. Presumably some form of Darwin Core.
Does anyone have a clue about the format. NBN clearly does it periodically with the entire Atlas (98 million) mine would be just a few hundred.

11

Re: GBIF

Hi Darwyn,
We share the datasets with GBIF as Darwin Core Archives (http://tools.gbif.org/dwca-assistant/), which are zip files that contain the metadata, the data schema and the data. I emailed GBIF helpdesk to ask how you should send your records to them. I will let you know when I hear back.

Sophie

Sophia Ratcliffe
NBN Atlas Data Manager

12

Re: GBIF

Hi Darwyn,

This is the information I got back from GBIF: For these publishers, I see two possible options:

    They could install and use their own publishing platform (the IPT is probably the easiest one)
    The alternative would be for us to help you/them find another host (someone from the GBIF network).

If it is only a few hundred records we could investigate sharing them from the NBN Atlas. We could also approach GBIF Spain (they have quite a few non-Spanish records and may agree) and I can submit them in Spanish.

Thanks, Sophie

Sophia Ratcliffe
NBN Atlas Data Manager

13

Re: GBIF

Thanks Sophie
I've had an enthusiastic response from GBIF's Jan Legind who seems to like the idea of sharing data through the Scratchpad infrastructure. Which would be very nice but I can't see the NHM developers countenancing that sort of module add-in, Charles Roper hasn't got anywhere in respect of his offer to collaborate with their team regarding FSC's Identikit keys on Scratchpads.
Some sort of international iRecord add-in would be super as there are many international Scratchpads (mostly "world" rather than my humble "europe") but I suspect we're in the realms of fantasy.
I don't think Spain would work either. It's very forward-thinking of them to upload outside their borders. The language barrier would be just too much.
I'd have to understand more about their IPT platform.
Recorder 6 of course supports recording outside the UK.
I suspect that there will have to be some more chat to get any progress on this. I'm in no hurry as I can just use GIS to hand-produce maps, the disadvantage of that is we're not contributing to the international databases that are so important to monitoring insect decline.
What's NBN's thoughts on helping out with non-UK occurrences? Aren't you close to this with non-OS places such as the Channel Isles? Checklists are a problem of course (though these would be in any Scratchpad - and GBIF's lists)
Demand for this isn't going to be great but if one scheme succeeds then others might follow.
No hurry, Sophie, it takes a long time to extract occurrences from papers.

14

Re: GBIF

Hi Darwyn,
There is a call for allowing records from outside of the UK and IoM on the NBN Atlas - to allow museums to share their natural history collections with GBIF.

We haven't investigated how this would work or look on the Atlas - would you be prepared to share your dataset with the NBN Atlas with a view to us passing it on to GBIF? Then we can see if there is anything that we need to do on the Atlas to improve non-UK record flow to GBIF. I'm thinking mostly about unmatched names. We won't be able to do too much because our funding is for the UK and the Isle of Man but I am sure that we will be able to do something,

thanks, Sophie

Sophia Ratcliffe
NBN Atlas Data Manager

15

Re: GBIF

sratcliffe wrote:

There is a call for allowing records from outside of the UK and IoM on the NBN Atlas - to allow museums to share their natural history collections with GBIF.

That seems the best way of doing things - eminently sensible. I have a not insignificant handful of non-uk records myself and at a bit of a loss at how to deal with.

(if it made things easier - it may not! - I'd be tempted to say that the data wouldn't even need to be displayed/accessible on the Atlas)

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

16

Re: GBIF

It does sound like we should be supporting getting non-UK records to GBIF.

If you can send me some records, we'll upload them and see how they look on the NBN Atlas - I suspect that the records will need to be available on the Atlas. At the moment all non-UK or IoM records are not assigned a country. We may as a minimum need to assign a country to each record, which should be okay.

Thanks, Sophie

Sophia Ratcliffe
NBN Atlas Data Manager

17

Re: GBIF

An update on a little progress I've made:
1. In order to encourage European workers to record rather than just post their pictures (mostly not geotagged) to identification sites and/or Flickr I wrote up the Flickr/iNaturalist method in my scheme's newsletter ( Newsletter 2 ) The bottleneck on iNaturalist is the same as that in iRecord - shortage of experts. Collaboration with a fellow expert might help raise records to ResearchGrade.
2. There's a records comparison of GBIF vs "all that are known" map in that newsletter for the only species I've been able to investigate to any depth. Unless you know someone else so obsessive as to obtain every paper, country checklist, posted image etc. then that's about the best comparison you're likely to come across.
3. Format of "some records" is a problem. Any old spreadsheet will do for QGIS and that's mostly what I've got. Could really use something better (I've a bust R6), if I can find the format I'd think about assembling an MSAccess database. Can you guide me to that information Sophie, I lost a lot of links in my autumn flood damage.

18

Re: GBIF

Hi Darwyn,
The guidance for sharing records is here: https://docs.nbnatlas.org/share-species … bn-atlas/. There is a csv template to download from that page.

Information on the individual fields: https://docs.nbnatlas.org/guide-to-the- … -datasets/

The guidance for all Darwin Core terms on TDWG's website is here: http://rs.tdwg.org/dwc/terms/. You can include any of these terms in your records, our template and guidance only covers the required and desirable fields for the NBN Atlas and some more commonly used terms.

Please let me know if there is anything that I can do to help,

Thanks, Sophie

Sophia Ratcliffe
NBN Atlas Data Manager

19 (edited by Darwyn Sumner 29-07-2020 07:50:38)

Re: GBIF

Well we succeeded. After a fair bit of work there is now a non-UK dataset uploaded to GBIF.
The NBN version is at https://registry.nbnatlas.org/public/show/dp246 and the GBIF one at https://www.gbif.org/publisher/1d7ce54a … 1a4a936776 (latter translated the GBIF taxon keys correctly, former only UK taxa)
A first for Sophie and her team and a first for a UK Recording Scheme.
Not as straightforward as it looks. To prepare the list of species occurrences I had to go back to first principles and construct a data model (simply Sources, Locations, Events, Occurrences plus a taxon dictionary using GBIF keys) using DwC terms in a spreadsheet. Then the NBN team converted that to the GBIF required format.
The system can only work through an organisation that is formally registered with GBIF so don't go thinking you can upload all records of your favourite taxonomic groups directly to GBIF, it would have to go through a registered Recording Scheme so you'll have to get them interested in direct GBIF uploads first.
There may be something of interest arising from the project to those who maintain personal lists of occurrences from abroad however. There's a DwC spreadsheet template known to work. That can be downloaded at http://www.micropezids.myspecies.info/node/357 I've used it to tidy up all my personal records from abroad, many happy hours flicking through IGN maps and Google Earth.
I did write it all up neatly as I went along but then messed the notes up with a lot of scribbling. Ask me if you want them.
No Recorder 6 involved.