1

Topic: Best report for spatial query

Hi,

I'm having a mental blank here and not finding the documentation I need. Please can someone point me in the right direction?

I have several report outputs which show records for a particular species or survey. Dragging a polygon (rectangle) on the map dynamically restricts the data shown on the grid. These use > Reporting form, Reporting page (customisable).

This is fine but what is the best approach to allow records to be extracted for a more precisely defined polygon area like a national park or a selection of grid squares?  i.e. a spatial query where the location is specified either by drawing a complex polygon on the map or by defining it first?

And how to download all selected records - even when the map is showing 10s of thousands of records, the download option seems to limit to 20,000.

Thanks.

2

Re: Best report for spatial query

Hi

If you are using reports with names that begin filterable_ and your form structure includes a [standard params] section to add a filter bar then you can use the "Where" section of the filter to choose from pre-defined locations.
E.g. https://www.brc.ac.uk/irecord/all-records

Alternatively you could use a report of your own with a location parameter that offers a choice of locations. The selected location would then be used in the report to do a spatial intersection.

To draw a complex boundary on the map I was expecting to just add drawPolygon to the list of Controls to Add to Map in the Other Map Settings. This is not working for me when I try it. I've stepped through the JavaScript and seen the control added to the map but then it doesn't show up. I'll not go any further with that for now.

If you look at client_helpers/report_helper.php report_download_link() you will see there is an undocumented option, itemsPerPage, which defaults to 20000.  You should be able to add an @itemsPerPage with your preferred value in your form structure near the @downloadLink=true which I imagine it contains. The limit is in there for a reason - perhaps server load or PHP timeout - and I have seen advice to use several downloads with date ranges applied in order to get back large data sets.

Regards,
Jim Bacon.

3

Re: Best report for spatial query

Hi Jim,

Thanks for your reply.

I did think of the filter bar but this is not working in my reports and I couldn't easily work out why. It gives a Type Error "c.prop is not a function" in some js files (see attached screenshot, from http://the-eis.com/atlas/?q=test). Is this likely to be related to the jquery version or something? I am sure the filter bar used to work.

I also tried the drawPolygon option last week and - like you - saw that it didn't do what I thought it would. I was using Chrome and it didn't show up at all. Testing now, I see that it does show up in Opera. I think there is something strange going on in any case: on my computer, the filter bar won't show in Opera but does show in Chrome, while the draw polygon icon does show in Opera but not in Chrome. But on my colleague's computer, in Chrome the filter bar does not show and the draw Polygon icon does show. (The brc link you gave works correctly in all browsers I tested.) I'm confused.

Thanks for the info on the ItemsPerPage. When I tried putting this in the [report grid] section, it changed the number of rows shown on the grid but not in the csv file. I see itemsPerPage is set to 20 in private static function get_report_grid_options and to 20,000 in public static function report_download_link. I'm not sure where to put it in the form structure to tell it to apply to the download and not the grid, I tried it in several places.

Thanks again for your advice.

Post's attachments

screenshot.JPG 122.83 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

4 (edited by Jim Bacon 03-11-2017 19:52:59)

Re: Best report for spatial query

Hi

https://www.nuget.org/packages/Fancybox/ says Fancybox 2.1.5 has a dependency on jQuery >= 1.6.1 and you are on 1.4.4 so you could be right about that. You know https://www.drupal.org/project/jquery_update can help you try another version. iRecord is on version 1.8.

Sorry, didn't spot itemsPerPage was used for both purposes. The form structure passes it to report_grid() which calls get_report_grid_options() followed by report_download_link(). Worse than that, itemsPerPage is unset before calling report_download_link(). It is effectively hard-coded when used in this context. I think it'll need some sort of coding to work around it.

Jim Bacon.

5

Re: Best report for spatial query

Hi Jim,

Thanks for your assistance.

Upgrading jQuery to 1.8 has cured most of the filter bar issues. The only one not working now is (typically) the one I need most, the 'Where" option to filter by spatial parameters. It gives "TypeError: wkt is undefined" referring to sites/all/modules/iform/media/js/OpenLayers.js:1558. Do you think this is a related issue or something different?

thanks, Alice

6

Re: Best report for spatial query

Hi Alice,

The OpenLayers.js file is a stripped down version which works on simple sites and saves bandwidth. A quick thing you could try is to enable the Indicia Openlayers module that is bundled with the IForms module and then go to the Configuration > IForm > Settings and set 'Type of OpenLayers.js file used' to 'Official Minimised Version'. That is just an optimistic stab in the dark but it is worth a try before delving any deeper.

Jim Bacon.

7

Re: Best report for spatial query

Thanks Jim. I did wonder about the openlayers module yesterday and whether it should be enabled...

I have tried enabling it and testing both the official minified version, and the debug version, but I still get these kind of errors: 'TypeError: a is undefined' or 'TypeError: wkt is undefined'. This is with jQuery 1.8. Trying a higher version breaks other functionalities.

What else can I try?

Thanks, Alice

8

Re: Best report for spatial query

Do you have a test page up that we could inspect?
Jim

9

Re: Best report for spatial query

Hi,

Yes: http://the-eis.com/atlas/?q=test

Thanks, Alice

10

Re: Best report for spatial query

Hi Alice,
That page is not showing filters at the moment so I can't reproduce the error you are describing.
Jim.

11

Re: Best report for spatial query

Hi Jim,

It does show filters for me on Chrome if I am logged in but not for an anonymous user while on Firefox it does show for an anonymous user. (And not on Edge or Opera.) This is part of the problem, I think... perhaps some permissions conflict or browser issue?

Thanks, Alice

12

Re: Best report for spatial query

Not showing filters for me when Anonymous on Firefox, Chrome, IE11.
I can see this is the intended behaviour from client_helpers/prebuilt_forms/dynamic_report_explorer.php get_control_standardparams().
Perhaps if you clear caches on Firefox it will stop fooling.

I've applied for an account so I can log in.

Jim Bacon.

13

Re: Best report for spatial query

Once logged in, using the Where filter, I drew a boundary on the map to limit records. On applying it, Ajax requests are made to the warehouse to update the display. The response to these requests is (in brief)

There was an SQL error:  function st_makevalid(geometry) does not exist

Not surpringly, this unexpected response leads to JavaScript errors. Referring to https://postgis.net/docs/ST_MakeValid.html, check whether your version of PostGIS >= 2.0.0.

Aside: having logged in the filters show in all 3 browsers.

Jim Bacon

14

Re: Best report for spatial query

Hi Jim,

Many thanks for your help, I appreciate it a lot.

Running SELECT PostGIS_Version(); in the database gives: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1

So you are right - I need to upgrade PostGIS? Does this imply I need to upgrade the warehouse (it is Indicia Version 0.9.1)? I'm not confident to do this which is why I have not done so.

Alice

15

Re: Best report for spatial query

Hi Alice,

That is a difficult question to give a definitive answer to. It depends on backwards compatibility in PostGIS and the functions of PostGIS used by Indicia.

I can see that the installation notes for Indicia were updated on 11 June 2015 to say that PostGIS > 2.0.0 is required (https://github.com/Indicia-Team/indicia … 6548a4e799) while version 0.9.1.1 of the warehouse dates from 05 August 2015. That is encouraging.

I have no recollections of any compatibility issues and I find no mention of any in the warehouse issue queue. We are using PostGIS version 2.1

The ideal thing would be to test it on a development system. If that is not an option, ensure you have a good backup.

I note from http://postgis.net/install/

To go from 1.* to 2.* you need to do a hard upgrade.

More about that at http://postgis.net/docs/postgis_install … rd_upgrade. There is more encouragement there where it says

If your database is really old or you know you've been using long deprecated functions in your views and functions, you might need to load legacy.sql for all your functions and views etc. to properly come back. Only do this if _really_ needed.

I think you can be fairly optimistic that you can avoid upgrading Indicia although you probably shouldn't put that off forever.

Jim Bacon.

16

Re: Best report for spatial query

Thank you very much, Jim. I will try to upgrade PostGIS for now.

Alice