1 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Sensitive records

I’ve been thinking that we should add a sensitive flag to the occurrence table, at the very least this would let people contribute records and tick the sensitive box so they don’t appear in any public reports (or we can ensure the resolution is reduced).

We may need a more complex solution than this but it would be a good start. E.g. we might want to make records sensitive if they are for a particular species or record type (e.g. a nest is sensitive, not a bird flying past). And this may also have a geographical filter. One idea might be to implement something like the triggers and notifications, where reports are defined to select new occurrences which meet certain criteria except in this case they get auto-tagged as sensitive. Or we could create a "sensitivity criteria" table which has the following structure:
id
taxon_meaning_key
external_key (for taxon - identifies a TVK)
designation
record_type_term_meaning_id (identifies the record type which is sensitive in which case we would have to add record_type_term_id to occurrence)
survey_id
website_id
geom (defines a polygon)

This would let a "sensitivity criterion" to be defined using any combination of species, designation, record type, survey, client website and geographic area. E.g. all nests could be marked as senstive by adding a row with just the record_type_term_meaning_id set to nest. Or a species could be defined as sensitive in a region by setting the external_key to the TVK, and setting a polygon for the region in the geom field.

In which case it may make sense to separate sensitive into 2 flags to differentiate between records the user contributed but wished to keep from public view, and records which are identified as sensitive by the system. Then the system criteria can be refined without affecting the user's choice.

Any thoughts on this would be much appreciated.

John van Breda
Biodiverse IT

johnvanbreda's Website

2 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: Sensitive records

My first thought was that there is already a sensitive flag in the occurrence table. I had a look and it is actually called confidential. It is described as "Flag set to true if this record is confidential, for example if a user has elected not to allow their entered records to be indicialy visible."

I would have thought that sensitive records should be marked as such on submission rather than as a scheduled task.

With your table, identifying taxa seems hard. The occurrence has a taxa_taxon_list_id which can link to a taxon but the taxon may not have an external key. Also, I don't know how you are able to ensure that the taxon_meaning table is reliably joining taxa table entries, Is this anticipating the taxa table becoming much more dictionary like than currently?

I also don't follow the nomenclature for the record_type_term_meaning. Would a record_types table be adequate like there is a titles table?

I wonder who would maintain sensitivity criteria. It sounds like a core admin role.

3 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Sensitive records

Agreed, I overlooked the confidential flag. We need to review how this is handled in the various reports and views though to see when confidential data should be filtered out or lowered in resolution.
If the taxon does not have an external key (~TVK) then the meaning Id is slightly better than using taxon_id or ttl_id because it means that the record is detected whichever name is used (e.g. if a synonym or common name is chosen). Also if the data is set up properly it allows different instances of a name from different lists or different names for the same thing to share the same meaning ID. This is how the model is intended to work, but until now the UI has not really supported this. v0.7 will change this a little because I have been working on the code which lets you copy taxa from parent lists into child lists - in which case you get a new ttl_id but for the same taxon and meaning_id. At some point it would make sense to provide tools allowing different lists to be 'linked' so the instances of names for the same species end up with the same meaning_id.

If the usage of record type becomes bound into the core in this way, then we would add a record types termlist and refer to it from the occurrences table using a foreign key to the termlists_term.id. By storing the meaning ID as before we get round the problems with different variants of a name.

John van Breda
Biodiverse IT

johnvanbreda's Website