1 Reply by Rob B

  • Rob B
  • Veteran
  • Offline
  • From: Edinburgh
  • Registered: 08-05-2006
  • Posts: 88

Re: How to actually use web services

Hi there,

I've recently been asked to contribute to a report entitled 'Biodiversity of SWT Reserves'. As we have a small (c55k) number of records within our R6 so far, I was going to run a few searches on the NBN gateway web services. However, even after reading the blurb, I'm still no wiser how to perform a search which includes information found on the home page.

Specifically, I want to get data from this question: "Which species occur on SWT Reserves that are included on each of these three lists:
A. The Scottish Biodiversity List
B. UK BAP list
C. Species of Conservation Concern

I know that the returning data will be vast, but that's what I've been asked to produce.

So, how would I do this?

At the moment,  I'm doing it on a species by species basis, but as we're talking 1500, you can appreciate why I'd like a more efficient way of doing this.

Many thanks now, Rob.

2 Reply by Richard Ostler

  • Richard Ostler
  • Member
  • Offline
  • From: UK
  • Registered: 11-05-2006
  • Posts: 21

Re: How to actually use web services

Hi Rob,

Using the web services you can get a species list for a single reserve and filter it by a designation list (e.g. BAP list).

To run this type of query you should use the Species List web service (http://www.searchnbn.net/library/webser … esList.jsp). You should set the following options for the query:
1. set the designation attribute to the designation list (e.g. BAP) to filter on ( for info http://www.searchnbn.net/library/webser … nation.jsp
2. use a KnownSite GeographicalFilter specifying the ID and dataset for an SWT reserve to create a list for. You can download IDs for known sites here: http://www.searchnbn.net/library/webser … nSites.csv - look for SWT in the siteType column.

You cannot:
- filter the species list using multiple designation lists - you have to run a seperate query for BAP, SOCC etc.
- filter across multiple sites (i.e. you can't get a species list based on all SWT sites using one query.

A final point, the species list returns records whose grid reference overlays the site boundary. Therefore you may get records which were actually recorded outside of the site boundary. Using the GeographicalFilter you can limit the record resolution to query on. By default records recorded upto 10km resolution will be selected. However, you can specify a maximum filter resolution (either 2km, 1km or 100m) to refine the list. You may this useful for small sites.

Hope this helps

Richard

[b]Richard Ostler[/b]
NBN Developer

Richard Ostler's Website

3 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: How to actually use web services

Rob,

The web services allow you to access the functionality of the gateway programmatically using a programming language such as Java, PHP, .NET, etc. via a technology known as SOAP (more technical article here.

Unless you're comfortable with programming in one of the aformentioned languages (or any language that has good support for SOAP), you're going to have real difficulties doing anything with the NBN Web Services. It think it's fair to say that these web services make life easier if you're a programmer, rather than an end user.

Charles

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

4 Reply by Rob B

  • Rob B
  • Veteran
  • Offline
  • From: Edinburgh
  • Registered: 08-05-2006
  • Posts: 88

Re: How to actually use web services

Hi Charles,

As usual, you hit the nail on the head in nice clear language. I'd suspected that I was missing a fundamental factor when reading the instructions, not least as the examples didn't really make much sense.

Ok, the manual method it is then!

Cheers now, Rob.

5 Reply by Nick

  • Nick
  • Veteran
  • Offline
  • From: Aberdeenshire
  • Registered: 11-05-2006
  • Posts: 70

Re: How to actually use web services

Hi Rob

So the trick is to find a friendly programmer who is willing to help :)

However, one thing that you may want to consider is what level of data quality you are looking for. For example, if you want to specify the exact boundary of the Reserve then you need to specify the polygon(s) that this covers and is probably a lot more work than a search for the general area(s) of the Reserve (i.e. simple point and buffer  - see the following test pages http://www.hoverfly.org.uk/NBN_SpList_Test.php and http://www.nesbrec.org/dbasesearch1.php for similar examples).

You would retrieve all the species but you could build local filters using data from various sources such as http://www.ukbap.org.uk/Library/SOCC6.XLS

Cheers

Nick

(ePlanning Project Manager) Aberdeenshire Council

Nick's Website

6 Reply by Richard Ostler

  • Richard Ostler
  • Member
  • Offline
  • From: UK
  • Registered: 11-05-2006
  • Posts: 21

Re: How to actually use web services

Hi Nick,

The SWT reserve boundaries are already loaded on to the NBN Gateway database. Therefore you can query the reserve boundary directly by its ID using the KnownSite GeographicalFilter. This is faster than defining a user defined polygon query because a lot of the query processing has already been done offline and prepared as summarised tables.

For performance and code simplicity, using boundaries already existing on the Gateway (known sites) is preferred to doing the equivalent userdefined polygon query.

Richard

[b]Richard Ostler[/b]
NBN Developer

Richard Ostler's Website

7 Reply by Nick

  • Nick
  • Veteran
  • Offline
  • From: Aberdeenshire
  • Registered: 11-05-2006
  • Posts: 70

Re: How to actually use web services

Hi Richard

Well that makes life a lot simpler and as you say will reduce the load on the server. I guess that coupled with existing designations (http://forums.nbn.org.uk/viewtopic.php?pid=448) this should be fairly quick to set up.

Cheers

(ePlanning Project Manager) Aberdeenshire Council

Nick's Website

8 Reply by Graham French

  • Graham French
  • Administrator
  • Offline
  • Registered: 23-05-2006
  • Posts: 430

Re: How to actually use web services

Hello Rob

It sound like you need to ask for a custom download from the NBN. By applying a known taxon list (in my case, dragonflies) to a list of polygons on the NBN Gateway (in my case NNR and SSSI's) Andy Brewer was able to quickly get me what I was looking for. Maybe he can help, I would certainly try him before you go down the manual approach

Best wishes
Graham

Graham French's Website

9 Reply by andyb

  • andyb
  • Member
  • Offline
  • Registered: 01-09-2006
  • Posts: 22

Re: How to actually use web services

Hi Rob,

Graham is right. A custom download is your best bet. This is part of the NBN Gateway service provision.

First of all, make sure you have the access to the datasets that you need by applying on the Gateway, if necessary. Obviously I can't send you data that you don't have download access to.

Then write a quick specification which says:
- what data you want (for example a list of all BAP, SOCC and SBList species that occur on SWT reserves including a start year, end year and 10km grid reference)
- what format you would like it in (within reason!)
- what your deadline is

Send the specification to me here: a.brewer@nbn.org.uk

It can take a couple of weeks for you to get your download because I have to fit it in to my other work. It also takes me a while to work out how to build the query!

Best wishes,
andyb

10 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: How to actually use web services

I didn't know about this service. So would I be right in thinking that it's possible to request all data for the VCs of East and West Sussex (given relevant permissions, of course). Is it also possible to request updates, e.g. only data added or changed since last request?

Charles

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

11 Reply by andyb

  • andyb
  • Member
  • Offline
  • Registered: 01-09-2006
  • Posts: 22

Re: How to actually use web services

Hi Charles,

We don't advertise this very much because we would soon be inundated by requests and we don't have very much time allocated for the service. In fact, it's just me fitting these downloads in with the rest of my work programme.

While we don't have a policy, as such, I see custom downloads as a means of providing people with data for particular projects or reports that would be time-consuming to obtain using the Gateway interface. Basically, I can run a query on the Gateway database so that a user, such as Rob, doesn't have to manually click through, copy and paste hundreds of site reports.

I'm not mad keen to help populate local databases on an ongoing basis for the 'usual reasons'. That is, it wouldn't be good to have different versions of national datasets floating around and used for different purposes. This is especially true if locally collected records are then integrated with them that never find their way back to the original data provider. A custom download is a disposable 'snapshot' of the database at a fixed time. The Gateway website will always be there (hopefully!) so that you can use the latest version of the dataset when you need it, which is what the NBN is about after all.

Having said all that, it would be straightforward to provide you with that particular download if you need it for a particular report or project.

andyb