1

Topic: Format for Atlas upload date field.

The Darwin Core template says for the "eventDate" field that:

Definition: The date or interval during which an Event occurred.

Examples:
2009-02-20 is 20 Feb 2009
1809-02-12 is 12 Feb 1809
1906-06 is Jun 1906
1971 is just that year
"2007-03-01/2008-05-11" is the interval between 1 Mar 2007 and 11 May 2008
"2007-11-13/15" is the interval between 13 Nov 2007 and 15 Nov 2007

For the purposes of exporting data from my database and generating the XLS/  CSV file required by the Atlas, what is the format of the "eventDate" field.  I am assuming that it cannot be a "Date" format field as records covering a range of dates will not be in a "real" date format.

2

Re: Format for Atlas upload date field.

Hi Matt,
Yes, the eventDate field can only be text field in the new Darwin Core format. I have not found it very easy to work with the two dates in the same field. It's okay if you run the script/export the data in the correct format and don't open the file again. But what I've found is that if you open file and then save it, there can be some problems with the dates, and you can't reformat as date once you have a mix of date types.

I am at a workshop next week and I will ask if there is anyway we can improve this.

Best wishes, Sophie

3

Re: Format for Atlas upload date field.

Thanks for that.  I think the best way forward will be to set up my database to generate text versions of my current Start and End Date fields and then merge them together before exporting them.  In practical terms, I think the easiest way forward for data providers would be for them to supply the NBN with Start and End Dates as before, and let NBN do the "fiddling" to generate this particularly obscure date format.   I'll wait to hear about your workshop.

4

Re: Format for Atlas upload date field.

Slightly off topic, but does the Atlas allow latest-dates to be specified (i.e. start date unknown)?  ISO 8601 date format doesn't appear to allow for this, but does NBN have a way to work around the problem?

Would a placeholder such as "0000-00-00/1977-06-14" be acceptable?

The old exchange format allow the start date to be open-ended, which is useful if only the date-of-publication or the recorder's date-of-death is known for an occurrence. Shoehorning such records into a fixed range can be quite misleading.

BSBI has several hundred thousand records where the start date is unknown.

Tom Humphrey
Database Officer,
Botanical Society of Britain and Ireland (BSBI)

5

Re: Format for Atlas upload date field.

I can't find the official reference for this but I believe Darwin Core uses '00' for an an unknown month and day - presumably year also.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

6

Re: Format for Atlas upload date field.

charliebarnes wrote:

I can't find the official reference for this but I believe Darwin Core uses '00' for an an unknown month and day - presumably year also.

Thanks, I agree that that is sometimes done, but the official specification doesn't appear to allow for it and in general missing day or month values would be omitted rather than zeroed, but that doesn't help with representing an open start date.

terms.tdwg.org/wiki/dwc:eventDate

I'll probably try sending data with the 0000-00-00 placeholder and see what the response is, but it would be good to know the correct approach, there must be other people with the same problem.

Darwin core also allows a descriptive verbatimEventDate - if that were supplied in addition to event date would the Atlas display it? That could allow a pairing of the a single end-date with a verbatim date that explained the date basis, e.g. "to 1977-06-14 (date-of-death)"

terms.tdwg.org/wiki/dwc:verbatimEventDate

The spam filter is too aggressive, so I had to defang both links both the post could be accepted :(

Tom Humphrey
Database Officer,
Botanical Society of Britain and Ireland (BSBI)

7

Re: Format for Atlas upload date field.

Hi Matt and Tom,
I am sorry for my delay in replying.

In reply to Matt, the only way to supply a date range is to use the start and end date in the same field, as you mention in your comment. I agree that this is not straightforward and if data partners would prefer to supply the start and end date in separate fields and include the date precision, I will format the date as required for upload.

Tom, it is better not to include the date rather than use a placeholder. So for a year range, where the start year is unknown, the entry should be (for example): '/1998' or '/1977-06-14'. Yes, the verbatimEventDate is also displayed. I have started using it for those records where the start date is unknown.

Just for information - if the event is on a single day, the day, month and year can be supplied in separate fields. I am still learning/figuring out the best darwin core terms to use and how the Atlas manages them.

Best wishes, Sophie

8 (edited by charliebarnes 26-02-2018 09:53:09)

Re: Format for Atlas upload date field.

but the official specification doesn't appear to allow for it

That's just ISO 8601 ("best practice" and not mandated?). AFAIK ISO 8601 does allow for missing elements of the date, replacing them with '-', but I've seen no reference to this in Darwin Core.

Also see the last comment on this (open) bug: http://dev.gbif.org/issues/browse/PF-2355

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

9

Re: Format for Atlas upload date field.

Hi Sophie,

Thanks, that's really useful. I've got various other queries about darwin core formatting of files for nbn import. Is this message board the best place to ask about those, or would you prefer email (or something else)?

regards,

Tom

Tom Humphrey
Database Officer,
Botanical Society of Britain and Ireland (BSBI)

10

Re: Format for Atlas upload date field.

Hi Tom,
Whichever you are most comfortable with. I am sure that your queries are useful to everyone, so the best place is probably here, but please email me directly as well, if that is easier. I am updating our help as well, to try and have a more comprehensive guide to formatting darwin core files for upload.

Thanks, Sophie

11

Re: Format for Atlas upload date field.

sratcliffe wrote:

In reply to Matt, the only way to supply a date range is to use the start and end date in the same field, as you mention in your comment. I agree that this is not straightforward and if data partners would prefer to supply the start and end date in separate fields and include the date precision, I will format the date as required for upload.

Thanks for that, if that could be included in the "Upload file specifications" then that might be helpful to people who are not sure if they have the ability to provide "odd" formats.

For ease of use, rather than having to go through all my records and check whether I have a different end date from a start date, I think supplying everything as aaaa-aa-aa/bbbb-bb-bb will be the essiest thing to produce.

12

Re: Format for Atlas upload date field.

Thanks Matt, yes, I am working through the help. Any comments on how to improve the help would be great.

The best thing is to try with the format aaaa-aa-aa/bbbb-bb-bb and see how it looks on the Atlas. The Atlas may recognise that the dates are the same. I can always re-upload if it doesn't work well, after formatting the dates.

13

Re: Format for Atlas upload date field.

Should it be a problem if both dates are the same?  If it will accept a date range in that format I would expect it should accept an entry that is effectively a single day as well.

14

Re: Format for Atlas upload date field.

I have never tried it and unfortunately I don't have a test system that I can easily try it out on. It will certainly accept a date range of a single day, I just don't know how it will look on the occurrence records. I don't think that it will be a problem.

15

Re: Format for Atlas upload date field.

Same subject, different date type.

The NBN Gateway would allow for a date type of "pre year XX", eg pre 1927 would have no start date, an end date of 31/12/1927 and a Date type of "-Y".

How should we format dates of this type for Darwin Core / NBN Atlas upload?

16

Re: Format for Atlas upload date field.

Hi Matt,

Sophie's earlier reply addresses that:

sratcliffe wrote:

So for a year range, where the start year is unknown, the entry should be (for example): '/1998' or '/1977-06-14'.

though my recent data import suggests that this isn't quite working correctly yet.

https://records.nbnatlas.org/occurrence … dde960689b

Tom Humphrey
Database Officer,
Botanical Society of Britain and Ireland (BSBI)