1 Reply by petereeles (edited by petereeles 13-01-2008 09:18:13)

  • petereeles
  • New member
  • Offline
  • Registered: 12-01-2008
  • Posts: 4

Re: Missing data from datasets

Hi all - I'm new to this forum so please be gentle with me :)

I'm trying to get butterfly species data at 10km resolution from the various datasets - especially "Butterfly distributions for Great Britain for the period 2000-2004 from Butterfly Conservation". I'm trying to ultimately generate distribution maps for all of the butterfly species.

However, when I use the OneSpeciesData web service for one of our commonest species (vernacular name Small Skipper, Taxon key NHMSYS0000504515, start:2000, end:2004), I get just 6 sites back:

SW771572
TL070390
TQ4109
SP922302
TQ228771
SU3903

For some other species, I get no sites at all! Any comments/thoughts etc. would be most appreciated. I must admit, I thought I was going to get hundreds of records back!

*** Update ***
Using the GridMap web service, the map shows the correct distribution and, after some digging, I've determined it's using all of the datasets. So I looked at the datasets being used in the response from the OneSpeciesData response, and it's only using 1 dataset (not the Butterfly Conservation dataset). If I then specify a dataset to use in my request (to "force" the use of the Butterfly Conservation dataset) I get no results at all, but don't understand why. So how come the GridMap seems to work OK but getting the raw data doesn't? Help!

Thanks in advance,

- Pete

2 Reply by JamesPerrins

  • JamesPerrins
  • Member
  • Offline
  • Registered: 10-05-2006
  • Posts: 32

Re: Missing data from datasets

Hi Pete,

I suspect you have had an introduction to the delights of gateway security - which takes a while to get your head around - but ultimately is all there for a reason (even if its a bit frustrating at times !)

For certain requests (and I think OneSpeciesData is one of them) you need to have download access rights to the data - as in theory you could download the records using that.

Others - such as specieslist (and I think the GridMap) - does not require download rights to the data (as the end result is a generalised position)

I think this is probably the cause of your problems (check and see if the few records you get back come from datasets where you - or public - have download rights) - to be honest I don't use the OneSpeciesData request that often - so could be wrong off the top of my head - but it sounds just like what you see with some of the other requests.

Workarounds are:
1) Use a different request (e.g. gridmap) if that meets your needs
2) Pursuade butterfly consrvation to grant you download righst to those records for your specific application - and then make secure calls to the gateway

Good luck

James

3 Reply by petereeles

  • petereeles
  • New member
  • Offline
  • Registered: 12-01-2008
  • Posts: 4

Re: Missing data from datasets

That's a good thought James - thanks. Even though I only want 10km resolution records to be included (to which I have access), I suspect most (all?) records are at a higher resolution than this and so they don't appear - and assume that there's no filtering performed to convert higher-res records to a 10km resolution (only) record. So, for example, a site with a grid reference of TG12345678 won't get converted to TG15.

I'll wait to see if anyone can confirm this as the problem and then, as you say, apply for access!

Regards,

- Pete

4 Reply by JamesPerrins

  • JamesPerrins
  • Member
  • Offline
  • Registered: 10-05-2006
  • Posts: 32

Re: Missing data from datasets

Yes - its not just access level - is specifically download rights that have caused complications in this area

For example - I can get a specieslist for all species in an area - but if I then ask for details of a particular species I can get no records - if I don't have download rights

5 Reply by petereeles

  • petereeles
  • New member
  • Offline
  • Registered: 12-01-2008
  • Posts: 4

Re: Missing data from datasets

Thanks again James. I made a request, as suggested.

Do you know how long it takes for a request to get approved / unapproved? I'd like to know either way :) Or does it depend on the efficiency of the dataset provider?

Thanks,

- Pete

6 Reply by JamesPerrins

  • JamesPerrins
  • Member
  • Offline
  • Registered: 10-05-2006
  • Posts: 32

Re: Missing data from datasets

Hi Pete,

As you suspect - I'm afraid this is entirely in the hands of the dataprovider - depenending who it is - it can be worth a phone call - sometimes you can find that the nominated contact for a dataset has moved on .....

Best wishes
James