1

Topic: EasyMap Web Service has been updated

We have released an update to the EasyMap Services that were previously hosted by BRC and have folded them into the gateway itself. This means that all websites using the EasyMap Service will now have a link to current data rather than using data from the old gateway.

This update was designed with the previous specification so there should be no changes necessary to keep existing calls to the service working and we are redirecting calls to the existing services hosted on brc.ac.uk to our service so no changes need to be made.

You can view documentation for the service here https://data.nbn.org.uk/Documentation/W … p_Service/

If you have any issues with the new service please let us know, but as long as your implementation follows the existing patterns there should be no issues.

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

2

Re: EasyMap Web Service has been updated

There certainly are 'issues', Matt.

The display of the maps that we have had showing splendidly for years (see http://www.hbrg.org.uk/FoHW/Conopidae/Conopidae.html, for example, and click any of the species names) is grossly distorted, and that is in Opera, Chrome and IE, and in W8.1 and Vista.

Please do some elementary testing and sort the aspect ratio of these maps.  At the moment they are embarrassing.

Murdo

3

Re: EasyMap Web Service has been updated

The issue is not that we did not test this (we did if you were wondering), we followed the specification exactly as written in the original documentation, which in this case relates to your website missing the h (height) parameter in your calls.

It is actually written in the original documentation that this defaults to 350 when no parameter is supplied, which we have faithfully recreated in this case. Obviously this is not what actually happens and I am trying to get in contact with the person who originally developed this so we can figure out exactly what they did.

In the mean time I am working to resolve this and I can understand how annoying this is but we cannot test for a feature or requirement that we did not know existed and was never documented.

Thank you for bring this to my attention,
Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

4

Re: EasyMap Web Service has been updated

Hi thanks Matt. A couple of things -

1. First couple of times I tried Murdo's link and also the DG moths I get a connection reset error. Hopefully I was just unlucky or you are doing work.

2. The DG moths site doesn't show a map, presumably because the NMRS isn't on the new Gateway? "Your request could not be completed because of the following errors. •No data returned for the criteria you have specified" This is a shame, as I always check it for each new moth I find in my VC72 garden but of course that issue was going to arise at some point when v4 is switched off. I'll let Mark at DGERC know there is an issue.

I assume that the Easymaps widget, since it is easy, doesn't involved logging in like the main web services, so will all calls made from this show up as being from BRC on the data providers stats?

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

5

Re: EasyMap Web Service has been updated

Hi Teresa, I have just released a fix for something else, so you might have just caught the service at a bad time in this case, the site appears to be working fine now however.

The NMRS dataset is on the gateway, however this system works on the same permissions as the rest of the gateway (you will see whatever level of access you have as a user, or public access if you are not logged in). In the case of the NMRS dataset (https://data.nbn.org.uk/Datasets/GA000591 I think) there is no public access to any data from the gateway, so the EasyMap Service itself wouldn't be useful in this case.

However you can get around this by having some sort of intermediary service that logs into the gateway (as a user with the correct level of access to this dataset) and requests the maps (these could be cached for use by your website).

The calls to this service aren't logged as no actual record data is taken from the gateway only a map in this case (we log downloads from the downloads wizard and any request that returns complete records from the API), I assume this is what you mean by data providers stats, but please correct me if I am wrong. In all respects every part of the gateway runs off of the web services so all services including this one are running off of the core web services including the web site itself.

Hope that makes some level of sense, please let me know if something needs further clarification.

Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

6 (edited by DGERC 23-07-2014 15:23:23)

Re: EasyMap Web Service has been updated

Matt

With regards to the maps using the easymap service on the DG Moths site, there are two separate issues. One is that, for the macro moths only, the dataset it is drawing from has no public access on the new gateway. At present there is no way around this issue.

The second, which affects all our local maps, is that the service appears not to be reading some of the parameters - specifically these are

&bl – bottom left corner of map area, given as a 10km square
&tr – top right corner of map area, given as a 10km square

These were added to enable the maps to focus on a specific area. I am aware that these parameters were not in the original published documentation but I reported this to the NBN on 20/12/2013 to ask that this be included in the re-write of the service for the new gateway, and was assured that this would happen.

An example of the usage of these parameters is given below

http://www.brc.ac.uk/schemes/NBNWidget/ … 0000501364

Please could you look into this so that at least our micro-moth maps will display correctly on the site.

Thanks

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

7

Re: EasyMap Web Service has been updated

Hi Mark,

With regards to the public only access, I did offer a work around, using an intermediary service logging into the gateway and caching the maps locally, but I understand this is still a very technical task and not simple and may require some negotiation with the dataset owners to make sure this use case is all above board.

The tr/bl feature has been carried over to the new service, the issue that you are having here is that not all grid references are currently in our databases. Early on in the design of Gateway V it was decided that we would not store grid references that have no records associated with them (they are created on record ingestion currently). In this case you have unfortunately chosen one such 10km square which was not imported, NW71, however NW72 is in the database as a quick fix in this case.

I apologise for this as this particular instance, it is a bit of an edge case however I agree it should work correctly. I will try and see if I can get the system to import at least all 10km OSGB squares today, but I can't guarantee if it will be resolved today, at best it will be fixed tomorrow when the data is replicated to the warehouses, I will try and keep you updated with my progress.

This issue however will continue to affect people using 2km and 1km grid references as bounding boxes as they are more likely to not exist (we create a tree upwards from a record so if a record is at 2km then we wont create the 1km or 100m squares but will create the 10km square for example). I haven't seen much use of this feature so I can't say with confidence what is most used here, however I would assume 10km would be the most prevalent use of this feature.

Once again I apologise for these issues and I am endeavouring to fix them as soon as possible, if you encounter new issues in the mean time please let me know.

Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

8

Re: EasyMap Web Service has been updated

Also I have added a quick fix for the issue raised by Murdo, while I try and figure out what exactly the old service used to do with regards to missing parameters, I have made it so it copies the value from the given height or width parameter to create a square if one is missing and only defaults to 350 if no height or width are given.

Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

9

Re: EasyMap Web Service has been updated

Thanks for tracking down the issue Matt. Doesn't matter if our moth maps are down for a few days, so long as the issue behind them will be fixed fairly soon.

I think that anyone that uses this feature is likely to use a hectad reference (in fact this could be specified in the Easymaps instructions) so that it should work for everyone in future once you get the 10km squares sorted on your database.

With regards to the other issue of changed access permissions, then whatever solution we look at is likely to mean quite a bit of work as the moth website is a series of static pages and each will need  the coding tweaked unless the dataset is granted public access on the V5 gateway, which seems unlikely at least in the short term. We'll find a way around this, but it won't be too technical as my skills and that of the county recorder don't run to that!

Cheers

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

10

Re: EasyMap Web Service has been updated

I see that the aspect ratio is now correct, Matt, so thanks for that.

With regard to the 'missing height' parameter (post above), I was advised several years ago by Jim Bacon when I first implemented EasyMaps, to include either height or width but not both.

Re testing, I would have thought that any proper testing regime would have checked that existing implementations of the widget were working properly.  The HBRG site, featuring as it does in http://www.nbn.org.uk/News/Latest-news/ … Maps.aspx, might have been an obvious one to check.  You might understand my irritation at having lost EasyMaps for 9 months or so, since that article was written by me at the request of NBN staff.

Murdo

11

Re: EasyMap Web Service has been updated

Syrphus wrote:

With regard to the 'missing height' parameter (post above), I was advised several years ago by Jim Bacon when I first implemented EasyMaps, to include either height or width but not both.

Presumbly the missing parameter was calculated to retain the correct aspect ratio?

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

12

Re: EasyMap Web Service has been updated

Matt

Any progress with the 10km squares issue? None of the maps on the dgmoths website are displaying at the moment. The micros should work when this issue is resolved as these are pulling data from a dataset with appropriate public access on V5 of the gateway.

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

13

Re: EasyMap Web Service has been updated

Hi Mark, I imported all OSGB 10km squares covering any landmass in the UK (excluding Ireland for now), these have been live for about a week now, so they should be working.

The example you gave above now works from here and as far as I can see the maps on the dgmoths seem to be working at least for me, if you are still having issues I will have a closer look into this, but as far as I can see its working.

Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

14

Re: EasyMap Web Service has been updated

Matt

One issue appears to be related to web browser security, as the new service seems to be returning an https rather than a http link which means that it is not displaying in Chrome.

When viewed in IE it works. However two issues here. Aspect ratio is now incorrect - I think Charlies something like Charlie suggested above must have applied as the square map is now horribly distorted. Also the 'terms' switch which I presume switched the text and link above the map on and off does not seem to be working any more. I see there is now a 'maponly' switch which we could use, but a fix for the 'terms' one would be better.

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

15 (edited by matt.debont 01-08-2014 13:11:58)

Re: EasyMap Web Service has been updated

Hi Mark, I am viewing the maps in Chrome without issue, the iframe is grabbing data from a https source which shouldn't be an issue, this is due to the fact that we no longer support http connections to the gateway. However grabbing https sources from a http page should not cause any issue with any browser.

The terms argument was not described in the specification that we were given so as such was never implemented, what exactly did it do in this case? Did it just remove the terms and conditions link? or all text above the map?

I will look into the aspect ratio in this case and try to figure out what is going wrong here.

Thanks for letting us know,

Matt

Edit: It actually looks like the terms argument was implemented, but its not working, I will try and figure out why, it looks like it just removes the text between the title and map, if that is what is expected in this case.

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

16

Re: EasyMap Web Service has been updated

Hi Matt

I think the 'terms' argument removed the all the text and link from above the map. Would be good if that were able to be fixed. I think the 'maponly' argument may have come along after we had implemented this so all our maps have the 'terms' argument in and set to '0'.

On Chrome the viewing issue is only when I'm signed in and I presume relates to the fact that I'm the administrator of the site (which is built using Google Sites). The maps do appear when I'm not signed in. Same issue with the county recorder who is also an administrator. Hence it shouldn't affect the general public just viewing the site.

Just tried out the species map web service on one of our pages too. Very good too, particularly with the date ranges you mentioned in the thread the other day.

Thanks for you help

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

17

Re: EasyMap Web Service has been updated

I have just pushed up a fix for this;

"terms" option should now work correctly and remove the text for terms and conditions as before, leaving the title intact

The aspect ratio should now be worked out when only a W or H parameter is provided, this comes from the bounding box being used so should always provide an undistorted map in these cases.

Hope that helps,
Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

18

Re: EasyMap Web Service has been updated

That's great Matt - seems to be working correctly now for our maps.

Cheers

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

19 (edited by DGERC 01-08-2014 14:48:00)

Re: EasyMap Web Service has been updated

One quick query Matt

Look at these two links - one returns a blank map and one returns an error. Both are macro moths so there is no data available to display in our region. Could you explain reason for the difference?

https://data.nbn.org.uk/EasyMap?w=300&a … 0000005994

https://data.nbn.org.uk/EasyMap?w=300&a … 0000005867

MARK

Mark Pollitt
SWSEIC (formerly DGERC)

20

Re: EasyMap Web Service has been updated

There are couple of things happening here, in this case there is no common name in the gateway for Rheumaptera hastata  (Linnaeus, 1758), so when you request a common name the easy map service gets very unhappy in this case, I will fix this to use the scientific name if no common name is available in the gateway.

From what I understand about the species dictionary, the common name is not a supported part of the species dictionary, it is something we add while processing the species dictionary, so in a fair number of cases there will be synonymy where a well formed name is available but it didn't meet the criteria to be automatically considered a common name for a given taxa. Hopefully that makes some sense, but the fix described above will be live in a few minutes.

Thanks again for bringing this to my attention and please do continue to let us know about these issues as we are happy to help and appreciate the feedback.

Matt

Matt Debont
Application Developer
Joint Nature Conservation Committee, Monkstone House, City Road, Peterborough PE1 1JY, UK

21

Re: EasyMap Web Service has been updated

Thanks again Matt - much appreciated.

MARK

Mark Pollitt
SWSEIC (formerly DGERC)