1

Topic: Date handling on atlas data

Records which are on the Atlas with a 'before year' date type eg. url=https://records.nbnatlas.org/occurrences/e8f08ef9-7883-4c3f-9996-f9554e0ed0c3 when downloaded have no date in the csv file.

Also I note there are some records on the Atlas seemingly with with no date at all https://records.nbnatlas.org/occurrences/ccfdfe9a-413b-4cf3-b03a-6f5a9396a35f.

I know R6 for example allows for the date to be entered as 'Unknown', which can be helpful in storing and retrieving vague info when managing a local dataset, but not sure that such data should make it to a national portal when one of the fundamental four W's is missing.

Mark Pollitt
SWSEIC (formerly DGERC)

2

Re: Date handling on atlas data

I'm not even sure if eventDate can handle absence of date?

https://www.gbif.org/developer/occurrence#p_eventDate

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

3

Re: Date handling on atlas data

Hi Mark and Charlie,
Yes, we have quite a few datasets that have no start date. I will contact all the data providers to see whether they can improve the quality of the date on these records.

eventDate (at least in the Atlas) can handle missing dates. I have raised the issue about whether we should allow records that do not have a date to be uploaded. I will write more once I have some feedback.

Thanks, Sophie

4

Re: Date handling on atlas data

Hi Sophie

Missing a start date but having an end date (i.e. a before date, e.g. -1960 in R6 terms) should not necessarily be an issue per se - it's an allowable format in Recorder 6 and I'm sure has been used by many to store records harvested from data which was included in a publication (so the publication date is known) but where the date of an individual record was not known. It's perhaps more an issue of how the csv download file portrays that. Getting no date information back at all for these records in the simple downloads seems odd.

On the other hand having completely missing date information as in the second example is much more questionable.

Mark Pollitt
SWSEIC (formerly DGERC)

5

Re: Date handling on atlas data

Hi Mark,
I hadn't looked at the simple downloads for these records. I will see whether it is possible to include some date information in them. Thanks also for the comment on completely missing dates. As soon as I have some feedback about this, I will let you know.

Sophie

6

Re: Date handling on atlas data

Is there any progress on the formatting of records with a 'before' date type in downloads? I've downloaded some data today and again there are many records with no date information in the csv file (simple format) - on checking one or two it appears that these are before dates

Without any date information in the download the records are worthless to any user. Including start and end dates separately would at least allow users to make something of the records extracted.

Mark

Mark Pollitt
SWSEIC (formerly DGERC)

7

Re: Date handling on atlas data

Hi Mark,
Improving the simple format csv download is on our priority development list. I have written a list of fields (and their sort order) for the download and I was going to share the list here, once I have checked it with our developer.

I am sorry for the delay in updating the csv file, hopefully we can resolve this very soon,

Sophie

8

Re: Date handling on atlas data

Thanks Sophie

That's good to hear.

Mark

Mark Pollitt
SWSEIC (formerly DGERC)

9

Re: Date handling on atlas data

Still a number of problems with missing date values in downloads for records that have dates on the Atlas (usually date ranges).
Example here - this has no date information in the simple csv file when downloaded.

Mark

Mark Pollitt
SWSEIC (formerly DGERC)

10

Re: Date handling on atlas data

Hi Mark,
Reuben is working on correcting the display of date ranges on the Atlas and in the downloads now. The start month and year are already included in the download and he is going to add an end date month and year as well.

Thank you for the example - it really helps to correct the problems. The was an error, in that the start year was ignored in the download for year range records.

Thanks, Sophie

11

Re: Date handling on atlas data

Is there any progress with adding end date month and year into the simple downloads? At present users who download records which have a year range (e.g. 1980 - 1989) can't get an accurate date details for the record if they use the simple download format.

Mark Pollitt
SWSEIC (formerly DGERC)

12

Re: Date handling on atlas data

Hi Mark,
I thought that Reuben was going to add end month and year to the download - but it looks like he only added the full end date, where it is available.

I will ask Reuben to add the end month and year as well.

Thank you for letting me know,

Sophie

13

Re: Date handling on atlas data

Thanks Sophie

Could someone post back when this has been done and is active?

I had a look at the full Darwin Core download format too and it appears that there are some records in downloads that don't provide any date information for the record at all. Can this be looked at too? It seems that someone needs to go through all the possible valid date formats (the R6 list of valid date types would be a good starting point) and make sure that someone downloading the data from these formats gets something meaningful and accurate in the download (whether simple or DC).

Mark

Mark Pollitt
SWSEIC (formerly DGERC)

14

Re: Date handling on atlas data

Hi Mark,
The start and end dates, plus individual fields for the start and end date day, month and year have been added to both the simple and full darwin core downloads.

Also, an 'LERC work' option has been added to the list of download reasons.

Best wishes, Sophie

15

Re: Date handling on atlas data

Thanks for that Sophie

Mark Pollitt
SWSEIC (formerly DGERC)