1

Re: on-line recording - an update

All,
I wanted to give you all an update on where we are with regard to the OPAL funded development of an on-line recording tool.
- Since the original meeting in Peterborough back in October 2007, I subsequently convened a smaller meeting to start to thrash out the detail of the requirements. These have subsequently been written up and can be downloaded from here http://forums.nbn.org.uk/files/OPAL Onl … entsV2.zip.
- There are at least two partners who are aiming to actually deliver a working system at the end of this financial year, namely, the Natural History Museum (who will be delivering a system on behalf of OPAL to capture the first OPAL survey) and the Freshwater Biological Association (who are aiming to deliver a system for river flies).
CEH will be contracting the majority of the build work for the first phase (up to the end of the financial year) to Biodiverse IT with work beginning almost immediately.
- The aim will be to work closely with the NHM and FBA over the next five months to ensure that, as far as possible, the initial core built meets their essential requirements. It is also hoped that it may be possible to use some development time from the NHM and FBA themselves to develop additional aspects to the system which are outside the essential core but enhance the functionality.
- In terms of communication around the project I propose to use this forum as this enables keeping a much broader range of stakeholders informed.
Best wishes
Steve

2

Re: on-line recording - an update

Thanks for the update Steve...

I noticed the following line in the RS: "In parallel with this the NBN has invested heavily in developing a suite of web services to allow users to create customised outputs from the NBN Gateway on their own websites.

Would you have a link to more information about this? I've not managed to find it via the NBN website, but its not a site I know well.

Cheers,

Steve

[b][email=%73teve@%73u%72rey-arg.or%67.uk]Steve Langham[/email] - Reptiles Officer & Webmaster[/b]
Surrey Amphibian & Reptile Group (SARG).

3

Re: on-line recording - an update

For more information on NBN Web Services see http://data.nbn.org.uk/library/webservices/wsIndex.jsp

4

Re: on-line recording - an update

Hello to all!

I've been developing a webgis based entirely on recorder database.

There’s already a public version with some minor bugs and is currently living at ---------> http://internal.bio3.pt/recorderlive

This version is a bit slow when running some queries, because all this data is being processed on the fly, with will end up delaying the application a bit.

By this approach which is powered by the recorder database without any middle DBMS or any tools besides of course the GIS platform behind it intended to understand some problems related to the delivery of the recorder data base online using a webgis.

The next step is working the integration and strategy of delivering this data over the internet,  in a faster and better way.

Regards.

[img]http://img90.imageshack.us/img90/1174/assinaturazx9.png[/img]

5

Re: on-line recording - an update

Steve,

Following your initial post on this, I wanted to suggest a couple of additions/amendments to the spec., and as you wanted communication through the Forum rather than direct, here goes!

Generally I think the outline is well on the right track now, and looks good.  I am very glad John is now involved with the build.

However, one thing struck me as apparently missing: the need to include 'habitat' as a standard field of data entry as part of species occurrence records.   So, in the section: "Data Entry Requirements", there is no mention of 'habitat' in any form on the spec for data entry card 1 or subsequent ones.   This must be the main obvious omission, particularly if we are trying to encourage recording organisations to improve the content and quality of the data they are recording.   By 'habitat' I would suggest that 'recording cards' ought to be able to use the kind of standardised biotope codes that are currently embedded in Recorder; as well as, perhaps, Keywords for micro-habitats generated by end users.

There are a few other minor queries and suggestions:

The spec. does not seem to make clear how an Administrator would add or change term lists in back-end databases and make these visible trhough the system.  Have I overlooked something here?

Is there going to be any automated system for updating master species lists from the NBN Species Dictionary?

Where comments are being made on a record by an external verifier, will the system ensure that information is maintained by the system on who made these comments and when (i.e., will there be a way of tracking these)?

When data are supplied as 'flat files' for download into other systems, will the Toolbox provide standard formats for these?

In Reporting requirements, could there be plans for a standard report on observations for a named vice-county or other standard geographical area?   Similarly, under Administration Requirements, as well as being able to define the zoom scale and centre point for maps, will the administrator be able to define geographical boundaries for mapping data?

In Database Design, for 'Occurrence: abundance', is it possible to include the capacity to record DAFOR and DOMIN scales, or vague numbers?   For Sample: I would suggest that grid reference or Lat.-Long. should be mandatory, not optional, even if these were determined by pointing to a map.   There is also no facility outlined to record source of data/reference/voucher specimen.     For Person: you need to have 'Title' and an optional entry capacity for recorder's address.  I would suggest that 'habitat' also needs to be an option alongside 'location'.

Regards,

Trevor

6

Re: on-line recording - an update

Hi Trevor,

Thanks for your detailed input - it's very reassuring to know that people are keeping an eye on what's going on! Here are a few responses for you:

However, one thing struck me as apparently missing: the need to include 'habitat' as a standard field of data entry as part of species occurrence records.

The core data model is designed to capture the minimum information required to make a biological record, and to allow extension for each website by providing custom attributes. The idea is we wouldn't want to enforce the presence of an attribute unless it is appropriate to all the different online recording schemes. Having said that, although I can see there might be a rare occasion where habitat is not of interest to a recording scheme, it is probably so ubiquitous that it's addition to the core model might make some sense. If anyone else has an opinion on this I'd be pleased to hear. Fortunately we are writing the Core Module in such a way that it will be quite trivial to extend with new attributes such as this, or even the addition of a new "dictionary".

The spec. does not seem to make clear how an Administrator would add or change term lists in back-end databases and make these visible through the system.

The Core Module will provide a number of services as you know, but in addition to the services it will provide a user interface that allows the site-level administrators to log on and create species/term lists by direct entry or uploading csv files. It will then be a task for the programmer of the online recording site to provide a control which links to the terms (for example an auto-complete text box). The control will use the services provided by the Core Module to access the data. As part of the project we will provide examples of how to do this so that the developers can re-use existing code rather than write new code.

Is there going to be any automated system for updating master species lists from the NBN Species Dictionary?

We will obviously intend this to be a simple as possible. Initially we are working on getting CSV upload working since that will accept data from pretty much anywhere, but given time I agree an automated update from the NBN Species Dictionary should be developed.

Where comments are being made on a record by an external verifier, will the system ensure that information is maintained by the system on who made these comments and when (i.e., will there be a way of tracking these)?

Yes, all comments should be tracked with at least basic metadata.

When data are supplied as 'flat files' for download into other systems, will the Toolbox provide standard formats for these?

Yes. Initially we are providing CSV standard support since that will cover most bases. The code is modular so it will be fairly simple to add new download file formats as required.

In Reporting requirements, could there be plans for a standard report on observations for a named vice-county or other standard geographical area?   Similarly, under Administration Requirements, as well as being able to define the zoom scale and centre point for maps, will the administrator be able to define geographical boundaries for mapping data?

Both of these are good ideas. I'll make a note for when we get that far on the list of features on the Google Code site.

In Database Design, for 'Occurrence: abundance', is it possible to include the capacity to record DAFOR and DOMIN scales, or vague numbers?

Yes, by adding a term list for the DAFOR/DOMIN values, and linking it to an attribute for occurrence records. Vague numbers could be handled by providing a text abundance attribute. As there are a variety of ways of storing this data I am not sure it is appropriate to try and include it in the core database model though.

For Sample: I would suggest that grid reference or Lat.-Long. should be mandatory, not optional, even if these were determined by pointing to a map.

I'm not sure I agree. There may well be a need for online data capture of historic or collections derived data where the spatial reference is not known, but a vague place name is given (as is accepted in Recorder). I think it would be a good idea to allow a site-level administrator to enforce additional mandatory attributes though.

There is also no facility outlined to record source of data/reference/voucher specimen.

Agreed, but I would expect custom attributes to be set up where these are required. However, I am open to the idea of including these in the core model as long as there is agreement they will be used by the majority of online recording schemes.

For Person: you need to have 'Title' and an optional entry capacity for recorder's address.

Agreed. I will add those.

I would suggest that 'habitat' also needs to be an option alongside 'location'.

The idea of storing a location for a person is two-fold. Firstly, it gives a possibility of displaying a person's location in any comments added to records, similar to many forums where you can see Avatars and other user profile information by posts and comments. Secondly, if we capture a spatial reference for a user, then we can default the display of maps to centre on their home, and provide simple data entry for observations made at home/in the garden (e.g. just provide an "at home" checkbox on the data entry page). So, I don't think habitat is applicable in the same way.

Phew! I hope that all made sense. Thanks for your input - I'll get on and make sure these ideas are listed on the Google Code issues list.

Best Wishes

John van Breda
Biodiverse IT