1

Re: Records with no location

I have come across a large number of records (>1000) that do not have a Sample Location. I would like to be able to fill in these blanks. Ideally I would like to do this based on whether records' grid references  are within polygons on a map layer, but suspect this may not be possible. Can anyone advise?

One problem is that I cannot work out how to run a query in Recorder to pick up the empty values. One possibility that has occurred to me would be to do this in SQL Maqnagement Tools and assign a dummy location to these entries, then use the dummy as a filter and manually assign locations back in Recorder. Any other suggestions?

Thanks

Gordon Barker
Biological Survey Data Manager
National Trust

2 (edited by RobLarge 09-08-2011 11:29:51)

Re: Records with no location

Not as simple a task as one might imagine (as I'm sure you have worked out). The initial selection would be easy enough as you could simply query the sample table for items with nothing in the LOCATION_KEY field (in SQL Server tools).

I would have thought though that the harder task would be to check for whether the sample spatial ref falls entirely within a polygon. A potentially very long piece of GIS data crunching, beginning with creation of a layer of all the records with square polygons reflecting their spatial ref precision, then performing "entirely within" query, using the results of that query to populate the sample location key in SQL Server tools again. What GIS system do you use?

I would have to ask what you hope to gain by it and whether you think it is really worthwhile. Might it not create a false sense of the reliability of the data? On our system for example we have hundreds of thousands of tetrad records which would not be attributable to any individual site (except in rare circumstances) even though they were in many cases recorded on accessible areas of species-rich land which we recognise as sites in our GIS layers, so if we wanted to query all of the records for a given site and be sure of getting them all, we still need to query all records which overlap and are thus potentially within the site. The situation is further complicated when site boundaries are edited from time to time. If a site is extended you would then have to requery and add any which fall within the new boundary.

I have given this a lot of thought over the years and always end up deciding that the only time we will associate an occurrence with a location, is when we can be sure that the recorder was familiar with the currently recognised boundary of the site and explicitly stated that it was within the site. Consider the situation where a recorder makes a minor error in the grid reference, giving a value which is inside a site rather than just outside. The spatial error is relatively inconsequential, moving the record by a few metres perhaps, but such post processing  as you propose creates a larger conceptual error by positively identifying a record as falling within the site, when it in fact does not.

I suppose, given that you work for NT it is possible that your dataset only contains data which were recorded within your sites though, which changes things somewhat.

Not sure if that helps you all that much though.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

3

Re: Records with no location

Hi Rob,

You are right, as we are dealing with discontinuous blocks of land, unlike a LRC, >99% of our records are associated with a NT property (or sub-site) in the location hierarchy and querying has been designed to work with this. So in order that these records are found by queries I need to fill in the blanks for the other 1%.

I would actually be using a polygon layer of 1K grid squares which helps to account for those marginal errors. I have this layer within the Recorder mapping and the polygons are associated with locations but I don't think there is any way to use this.

It would be a relatively easy task in MapInfo, but I am not confident about the exporting and re-importing required

Gordon Barker
Biological Survey Data Manager
National Trust

4

Re: Records with no location

On reflection I think I will move all the relevant samples into a survey set aside for them and work on them there. Does this code look like it will do the job:

update     SURVEY_EVENT set  SURVEY_EVENT.SURVEY_KEY ="DSS000460000004J"
FROM         SAMPLE INNER JOIN
                      SURVEY_EVENT ON SAMPLE.SURVEY_EVENT_KEY = SURVEY_EVENT.SURVEY_EVENT_KEY
WHERE     (SAMPLE.LOCATION_KEY IS NULL)

Gordon Barker
Biological Survey Data Manager
National Trust

5

Re: Records with no location

Ok how about this as a rough idea

Do the selection on the sample table in management studio

e.g  Select sample_key, spatial_ref from sample where LOCATION_KEY is null (not sure of the exact syntax here, but it shouldn't be hard to work out).

Import the results into Mapinfo, make it mappable and do a geographical intersection query with your polygon layer so you end up with a table which has both the sample keys and the location keys for the corresponding sites(this assumes that your polygon layer has location keys in it (ours does). If not you might have to do some more manipulation to get the location key from location name

Import the result back into management studio as a new temporary table within the recorder db (or elsewhere if you feel able to handle multiple databases at one time)

(back up the database first)
Create an update query in management studio of the form:

UPDATE SAMPLE INNER JOIN TempTable
ON SAMPLE.SAMPLE_KEY = TempTable.SAMPLE_KEY
SET SAMPLE.LOCATION_KEY = [TempTable].[LOCATION_KEY],
SAMPLE.CHANGED_BY = "Your recorder NAME_KEY",
SAMPLE.CHANGED_DATE = Now()
I would be tempted to also update the sample.comment  field with a note saying that you have made the changes

A similar update query might be used to also update the SURVEY_EVENT.LOCATION_KEY if appropriate (I think it would be) although that would require an extra join in the query.

then delete the TempTable

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

6 (edited by RobLarge 09-08-2011 14:35:55)

Re: Records with no location

Your update query has the wrong syntax I think, try this

UPDATE SAMPLE INNER JOIN SURVEY_EVENT ON SAMPLE.SURVEY_EVENT_KEY = SURVEY_EVENT.SURVEY_EVENT_KEY
SET SURVEY_EVENT.SURVEY_KEY = "DSS000460000004J"
WHERE (((SAMPLE.LOCATION_KEY) Is Null))

Although I don't think I would bother moving them to another survey, they will be impossible to put back afterwards.

By the way it is always good practise to update the changed_by and changed_date fields when you do this (and perhaps the comment too)

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

7

Re: Records with no location

Oh yeah and you should perhaps also check whether you are the custodian of the data you are updating in order to prevent possible duplication later.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre