1

Topic: Defra Species Surveillence project - setting up Indicia for monitoring

Jim/John,

Following my email to you (although this will be posted before that).  I am putting all my questions etc in the forum, so that any issues I have are available for the community.

First couple -

1. I think I have set up a website agreement in the Warehouse to access and verify all our records in Irecord, including those already submitted to the warehouse.  I have updated all our survey species lists to include the NBN taxon version key, as David Roy said I needed to allow the agreement to work.  How do I a) know where to access this information in irecord to check it is on there and b) what login should I use to access - can I use my existing irecord password?

2. I have uploaded the new Iform module and the Indicia feature module that Jim sent me.  How do I enable the features one, as all the boxes are unticked in the modules list in my drupal and I can't tick them.  Connected to this, does installing/uploading the Indicia features module have all the functionality I need to do all the instant indicia things detailed in the advanced training manual? The reason I ask is that your colleague on the training day last week suggested that I install/upload the instant indicia and then I will have all the functionality I need, but obviously I have installed Indicia features module already and Instant indicia has all of the folders that make up a drupal instance....which I don't need, apart from what is relevant to indicia. 

Any thoughts? Thanks for your help.

Sam

2

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam
1. a) What do you mean by "check it is on there"? The website agreement data stays on the warehouse and is used in report generation, so you can't check it is on irecord specifically. You could however submit a record and look to see if it appears in iRecord?
b) If the agreement is set up, then yes, you can just use a normal login to iRecord to see the records.
2. Normally, to enable features you go to Site Building > Features on the menu, not the modules list. But if you cannot tick a module, it may be because of a missing dependency (it will be shown in red if this is the case).

If you wanted to use the 0.8.2 update for Instant Indicia, you can download it and copy across just the iform and indicia_features folders from the modules folder, plus there is a folder called iform under sites/all/libraries. You can copy the contents of this into your modules/iform folder as well.

Cheers
John

John van Breda
Biodiverse IT

3

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam

A small extra note.
Your normal iRecord login should allow you to see your records but you may need additional privileges if you want to do verification. David has been administering this to date. There will be a Verify menu item if you have verification privileges.

As well as ensuring the NBN TVK is included in your species list, for proper operation with iRecord you need to be using matching taxonomic group values too. Did David mention that?

Jim.

4

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Thanks both.

John,

1a) Yes sorry that is what I meant - checking whether a record appears on irecord.  So hopefully I should see the NBIS website records on there then.  I was not sure if I could view the records under "my records" when logged in to irecord, but I guess I would not be able to as they are not records that I posted through irecord.  As Jim says I will need to get verification privileges to go through the verification process with our records. Which we need to do.  I will speak to David on that one.

[Jim - David did mention taxon group being needed. We have selected a taxon group from the drop-down for each species in each species list, is this what you mean?]

1b) Ok, I guess I just zoom to Norfolk to check out what is displayed and if they look right then the agreement must be working and if they don't, then it must need some adjustment.

2.  Yes I tried site building> features, but features does not exist on my menu, so I assumed the fact the module could not be enabled meant something is wrong.  I think maybe your suggestion of installing the relevant modules and folders from instant indicia may work better and hopefully solve that problem.  Either way, it seems I am missing one or two dependencies, so this could also need fixing anyway.

Thanks

Sam

5

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam
Yes, you won't have the site building > features menu item unless you've installed the features module. As your setup predates Instant Indicia, this won't be done by default. If you copy the required modules into place from the Instant Indicia 0.8.2 download then install Features, you should be able to use this tool to add other Instant Indicia functions (Easy Login is the one which will let your "My Records" work across sites). It will tell you if you need to install dependencies.
Cheers
John

John van Breda
Biodiverse IT

6

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam

In iRecord a user can select a taxonomic group that they have an interest or expertise in. (I'm a little vague about all the ways it is used.) What will then happen is that queries will be executed that select records by taxonomic group.

If iRecord selects records for species in the taxonomic group "Fungi" but you have invented your own taxonomic group called "Norfolk Fungi" then there will not be a match so your records would not be shown by iRecord. (Again based on very superficial knowledge of iRecord.)

Regrettably for me, almost every species list I have created uses non-iRecord taxonomic groups and I need to go back and update them all. At least I have direct access to the database and can do it with SQL rather than edit every taxon.

I have also pointed out some inconsistency in the taxonomic groups that iRecord is using (which are based on but not quite the same as those used by the NBN) which David may be resolving.

It would be good if the taxon groups that are being used by iRecord could be marked in the warehouse user interface.

Jim.

7

Re: Defra Species Surveillence project - setting up Indicia for monitoring

hi Sam,

iRecord uses a set of taxon-groups for users identifying groups they are interested in.  These taxon-groups are also used to assign verification roles (e.g. an expert having a verification role for 'insect-moth').  Therefore when setting up your surveys, if you want to benefit from the iRecord verification experts, you will need to match your species list to the iRecord taxon-groups.  iRecord uses the 'UK Master List' species list derived from the NHM/NBN UK Species Inventory, together with its taxon-groups.

Data from your NBIS surveys is appearing in iRecord reports, so the agreements have been setup ok.

best wishes
david

8

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Thanks John, Jim and David

John - you were correct, I needed to install some of the other modules that instant indicia uses, including features (as well as indicia_features) and a number of other ones.  I'm sure I will find I'm missing some, but at least I have easy login working.

Jim/David - thanks for the heads up on irecord. Could you let me know what the irecord taxon-groups are, so I know to only assign species in species lists in the warehouse to these? Thanks.

David - thanks for checking that our surveys are appearing in irecord reports.  I think I need to have a more intensive "play" with irecord to work out ho best to see our surveys, that is before I get any verification privileges.  Talking of this, what is the best way of us setting up NBIS extra privileges?

Thanks all

Sam

9

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam

If you login to iRecord and click on the My Account link at the top then the Edit tab then the preferences sub-tab you will see the list of taxon groups. Note case and spelling carefully. iRecord is, for example, currently using the "Reptiles" taxon group and not "reptile" which is also present on the Warehouse1. It is an untidiness that is offending me.

With regard to setting up verification rights, are you going to have a cohort of local experts that you will involve in verifying records submitted to your surveys? We might want to create an NBIS manager role for you and then an NBIS verifier role that you could give to your buddies. I'm thinking about how we could delegate the administration of your volunteers so David doesn't have to do it (or, worse still, asks me to do it). There is a Role Delegation module that would allow this.

Jim

10

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Jim,

Thanks for the taxon group info, that will allow me to check current surveys and no what to use in the future.

Re: verification rights.  Yes we will have a cohort of county recorders who will be doing this (and not us, other than setting up the rules for what should be auto-verified) and yes it would ideal if we could have a manager role.  A delegation module idea, sounds ideal to me....we would be very happy for that to be the situation.  The only thing is that David said that only national schemes and societies would be allowed to assign verifiers, do I misunderstand this?  I'm still a little confused where the data goes when verified on irecord, does it just stay on the warehouse with a verified tag, or does it go direct to NBN or both?  I assume it is both.  If so, what dataset would it appear under on the gateway, would it be under NBIS records or eg. Mammal Society.

thanks

Sam

11

Re: Defra Species Surveillence project - setting up Indicia for monitoring

hi Sam,

Verification roles in iRecord can currently be restricted in three ways.  1. by geographic area (currently only a single VC at a time - it is on our list of development to extend to multiple VCs and other geographic boundaries); 2. by taxon group (currently NBN species dictionary groups, although we are looking to improve this); 3. by Indicia survey (e.g. by NBIS surveys).

For your surveys, data is entered through your own recording survey forms and you use iRecord for verification.  Therefore, you can identify your own verifiers for your survey data.  However, there is considerable mutual benefit in identifying verifiers who will cover Norfolk in general and review data recorded through iRecord and your NBIS surveys.   At least for the time being, we are being cautious about setting up verifiers within iRecord - i.e. it is all coming through me!  The basic approach is to check they have a recognised role for national schemes and/or local record centres.  Jim is right that this isn't a scalable solution and Drupal may provide better solutions, but for the time being we will stick to the current approach.

As to data flow.  For data submitted through iRecord, we are identifying the preferred data flow on a group-by-group basis.  But , for you own NBIS surveys, this is in your control.  But the 'spirit' of using the shared warehouse facilities provided by BRC is a default position of open-ness where possible and regular supply to the NBN Gateway.  One option open to you is to let iRecord handle the onward data flow (e.g. to the NBN Gateway for verified data), as part of datasets for particular taxon groups.  For example, dragonfly data collected through your NBIS surveys could be packaged up along with other dragonfly data supplied to the NBN Gateway from iRecord.

all the best
David

12

Re: Defra Species Surveillence project - setting up Indicia for monitoring

David,

thanks for your response.  That is very useful.

RE: verifiers - agree with all that you day below.  Also agree that it would be good to have verifiers who cover Norfolk in general, although for some it maybe two VCs, as Norfolk has that issue; although for some it would be preferable, as our flora recorders are 2 separate people, one for west and one for east Norfolk.  We will need to run this all by them to get an agreement to be the verifiers.  We would then need to talk to you to agree the best way to set it up, including auto-verification (or whatever the correct terminology was).

Data flow - you suggestion for this option for us, i.e. irecord handling onward data flow to the NBN for verified records would be really useful I think.  Save everyone time.  We are definitely in agreement with you that all data through indicia and irecord should go straight to the NBN once verified.  Although I think the packaging up, e.g. with Dragonflies, makes sense to me, but I think it may have the odd object from Martin or recorders, if records through our website don't appear as NBIS records on NBN....even though we will have the data anyway.  This, I'm sure could be ironed out.

Anyway, thanks for the heads up on all this and we will be in touch very soon recording getting verification set up.  I'm concentrating on the form building first of all for our Defra project.

Thanks

Sam

13

Re: Defra Species Surveillence project - setting up Indicia for monitoring

hi Sam,

Let's keep in contact as things develop.  One small comment on 'auto-verification' - we don't (at least for now) do any auto-verification.  What we do is to use the NBN Record Cleaner rules to flag up 'unlikely' observations based on known ranges, dates within the season, ID difficulty etc.  This is with the idea that verifiers can filter the records to those most requiring scrutiny.  We are currently working with 12 species experts to derive rules for a number of species groups (mostly invertebrates).  The other thing we are developing in the next few months is the ability to identify 'trusted-recorders', i.e. those whose records may be flagged as 'auto-verified' if they do not fail any other checks.

In all the focus on verification we are aiming to provide tools to help verifiers but without sacrificing data quality, and to provide transparency in verification decisions that are made.

all the best
David

14

Re: Defra Species Surveillence project - setting up Indicia for monitoring

David,

Yes, now you mention it you did say this when we met in Cambridge about using flags, its all coming back to me....!

The flags and the 'trusted-recorders' ability will definitely benefit us and the speed of verification for our verifiers.

Thanks

Sam

15

Re: Defra Species Surveillence project - setting up Indicia for monitoring

John,

Quick question.  This is a problem I have had before actually:

I have installed all the instant indicia modules and folders that I believe I need, this includes CKeditor.  This has caused a problem which I have had before (well I assume it is CKeditor that is doing this) - when I click "load settings" in an indicia form nothing happens.  I no longer can untick the module, as it is now permanently enabled and I don't want to delete (or for that matter untick) as it is a dependency for the indicia quickstart feature.  Can you think of why CKeditor may do this and how to fix this problem or should I delete the CKeditor module?

I have attached a word document to my email to you with screen grabs of firebug which shows the error that comes up when I click on load settings.  The first is the error firebug shows straight away and the second is me clicking on the blue hyperlink on the right (the line of the error in js).

As I say, I'm pretty sure I had this before and the answer was to unenable/untick the CKeditor module, but as this and the indicia quickstart have so many dependencies within the instant indicia features/modules, it would be preferable not to have to do this.

Any thoughts? Much appreciated

Sam

16

Re: Defra Species Surveillence project - setting up Indicia for monitoring

John,

I unticked ckeditor and indicia quickstart and cleared all the cache, but unfortunately this did not solve the problem.  I have now returned the configuration to when I messaged/emailed you last.

Thanks

Sam

17

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam,
I don't think this is anything to do with CKEditor or Indicia Quickstart. From one of your screenshots, I noticed the following clue:

  <h1>Internal Server Error</h1>
  <p>Directory "/websites/123reg/LinuxPackage99/nb/is/_o/nbis.org.uk/public_html/sites" is writeable

I wonder if this is a server setup issue, though why it should occur at this point I've no idea. There are some tips here. Under the iform module, you only need to make client_helpers/cache and client_helpers/upload writeable.

Cheers
John

John van Breda
Biodiverse IT

18

Re: Defra Species Surveillence project - setting up Indicia for monitoring

John,

Brilliant, thanks.  For the moment I have changed my "sites" folder permissions to 775, as it was 777 (fully writeable).  This seems to have solved the problem.  I will look again at the iform folder permissions if I have further problems.

As you say, I have no idea why this has suddenly happened.

Cheers

Sam

19

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Ok. It might be worth checking that the upload and cache folders are writeable though.

John van Breda
Biodiverse IT

20

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Ok will do John, sounds sensible
thanks
Sam

21

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Jim,

I am putting together the forms now, starting with the location one and working through.  Would you be able to email or place here the code you have used in your form structure (user interface) for your locations form, environment form and one or two of the species forms used on Pondnet site?  It is so I can check a couple of things, plus it would be a useful reference point too.

Many thanks

Sam

22

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam
Here is what I did.
Jim Bacon.

Locations

=Pond=
[location name]
@helpText=Use the name shown on the top of your PondNet site map and indicate if the pond is the Focal Pond, or one of the other numbered ponds. <br/>E.g. SU4754 Windle Hill Focal Pond or SU4754 Windle Hill Pond 5
[location type]
@terms=["Site"]
[spatial reference]
@label=Grid reference
@helpText=Zoom in on the map and click the location of the pond to get the grid reference. You may type it in if you  know it.
[place search]
@helpText=To zoom the map to a place, type a town or village name and click the button.
<div id="map-info"></div>
[map]
@helpDiv=map-info
@helpToPickPrecisionMin=100
@helpToPickPrecisionSwitchAt=100
@hlpImproveResolutionSwitch=We've switched to a satellite view to allow you to locate your position even better. Click the + on the right of the map to change back.
@scroll_wheel_zoom=
@clickZoom=true
@fillOpacityGhost=0
@strokeOpacityGhost=0
<hr/>
[location comment]
@label=Notes
[*]

Enivironment

=Environmental Data=
[date]
@helpText=Select the date of your survey
[location map]
@fillOpacity=1.0
@scroll_wheel_zoom=
@clickZoom=true
@helpText=Select your location from the drop down list or by clicking it on the map. If this is a new location it must be added to the list first.
[smpAttr:191]
@label=Pond gone
@helpText=If the pond no longer exists, tick here.
[smpAttr:192]
@label=Pond new
@helpText=If the pond is new, tick here.
[smpAttr:193]
@label=Pond age
@helpText=If the pond is new, give approximate age.
[smpAttr:162]
@label=Pond area (m<sup>2</sup>)
@helpText=This is the surface area of the pond when water is at its highest level (excluding flooding events): usually in the early spring. If the pond is measured at another time of year, the high water level should still be evident from vegetation types and evidence of a drawdown zone around the pond.
[smpAttr:163]
@label=Pond dries?
@blankText=Select...
@helpText=Deduce pond permanence from local knowledge and personal judgement. The landowner may know how often a pond dries. However, if not, then try and make a judgement based on water loevel ath the time of the survey. Ponds that dry out annually usually have a hard base.
[smpAttr:164]
@label=Pond overhung (%)
@helpText=Percentage of pond overhung by trees and shrubs.
[smpAttr:165]
@label=Margin overhung (%)
@helpText=Percentage of pond margin shaded to at least 1m from the shore.
[smpAttr:166]
@label=Waterfowl impact
@blankText=Select...
@helpText=<strong>Major</strong> = severe impact of waterfowl i.e. little or no evidence of submerged plants, water turbid, pond banks showing patches where vegetation removed, evidence of provisioning waterfowl.<br/><strong>Minor</strong> = waterfowl present but little indication of impact on pond vegetation, pond still supports submerged plants and banks are not denuded of vegetation.<br/><strong>None</strong> = no evidence of waterfowl impact (moorhens may be present).
[smpAttr:167]
@label=Fish presence
@blankText=Select...
@helpText=<strong>Major</strong> = dense populations of fish known to be present.<br/><strong>Minor</strong> = small numbers of crucian carp, goldfish or stickleback known to be present.<br/><strong>Possible</strong> = no evidence of fish, but local conditions suggest that they may be present.<br/><strong>Absent</strong> = no records of fish stocking and no fish revealed during survey(s).
[smpAttr:194]
@label=Amphibians present
@helpText=Tick if present
[smpAttr:195]
@label=Amphibians description
@helpText=Record, as far as possible, amphibian species, life stage and approximate number.
[smpAttr:168]
@label=Ponds in square
@helpText=Number of ponds within your 1km grid square (1:25000 maps) 
[smpAttr:169]
@label=Ponds in radius
@helpText=Number of ponds within 1km of the pond (not separated by barriers to amphibian dispersal).
[smpAttr:170]
@label=Inflow
@helpText=Tick if present.
[smpAttr:171]
@label=Outflow
@helpText=Tick if present.
[smpAttr:172]
@label=Water clarity
@blankText=Select...
@helpText=Estimate turbidity by looking down into c.20cm depth of water in the pond.
[smpAttr:173]
@label=Aquatic vegetation (%)
@helpText=Percentage of pond surface occupied by macrophyte cover (between March and the end of September)
[smpAttr:175]
@label=Water left (%)
@helpText=Percentage of water area present in the pond relative to maximum water level.
[smpAttr:174]
@label=Drawdown (cm)
@helpText=Height difference between maximum and current water level.
[smpAttr:176]
@label=Grazing
@helpText=Tick if there is evidence that the pond is grazed by livestock.
[smpAttr:177]
@label=Grazing (%)
@helpText=Percentage of pond perimeter grazed.
[smpAttr:178]
@label=Grazing intensity
@blankText=Select...
@helpText=Rank on a scale of 1-5
<div id="pondmanage">
[smpAttr:179]
@label=Pond management
@helpText=Tick boxes for management within the last 12 months
</div>
[smpAttr:180]
@label=Other pond management
@helpText=List or add detail
<label for="landuse">Surrounding land use:</label>
[species]
@id=landuse
@rowInclusionCheck=hasData
@colWidths=[50, 25, 25]
@helpText=Estimate the percentage of surrounding land-use in distance zones from the pond perimeter
[smpAttr:181]
@label=Terrestrial habitat
@blankText=Select...
@helpText=Within c250m of the pond, but not including areas on the far side of any major barriers.<br/><strong>None</strong> = clearly no suitable habitat within immediate pond locale.<br/><strong>Poor</strong> = habitat with poor structure that offers limited opportunities for foraging and shelter (e.g. amenity grassland).<br/><strong>Moderate</strong> = offers opportunities for foraging and shelter but may not be extensive.<br/><strong>Good</strong> = extensive habitat that offers good opportunities for foraging and shelter completely surrounds pond e.g. rough grassland, scrub or woodland.
[smpAttr:182]
@label=Water quality
@blankText=Select...
@helpText=<strong>Bad</strong> = clearly polluted, only pollution-tolerant invertebrates, no submerged plants.<br/><strong>Poor</strong> = low invertebrate diversity, few submerged plants.<br/><strong>Moderate</strong> = moderate invertebrate diversity.<br/><strong>Good</strong> = abundant and diverse invertebrate community.
[smpAttr:183]
@label=pH
@helpText=
[smpAttr:184]
@label=Conductivity (µScm<sup>-1</sup>)
@helpText=
<label for="container-sample-photo_pondphoto"></label>
[sample photo]
@id=pondphoto
@caption=Photographs
@resizeWidth=1024
@resizeHeight=768
<p class=helpText>Click the Upload Photos button and select up to 4 photos from your local system. You may add a caption to each photo. Where possible, photos will be automatically resized on upload to 1024x768. This should ensure file size limitations are not exceeded.</p>
[*]

Amphibians

=Amphibian Survey=
[date]
@helpText=Select the date of your survey
[location map]
@fillOpacity=1.0
@scroll_wheel_zoom=
@clickZoom=true
@helpText=Select your location from the drop down list or by clicking it on the map. If this is a new location it must be added to the list first.
[smpAttr:197]
@label=Methods used
@helpText=Tick all methods used.
[smpAttr:198]
@label=Start time
@helpText=<em>hh:mm</em> using 24-hour clock
[smpAttr:199]
@label=Finish time
@helpText=<em>hh:mm</em> using 24-hour clock
[smpAttr:185]
@label=Air temperature (°C)
[smpAttr:186]
@label=Water temperature (°C)
[smpAttr:172]
@label=Water clarity
@blankText=Select...
[smpAttr:196]
@label=Rain
@blankText=Select...
[smpAttr:187]
@label=Wind
@helpText=Tick if wind was disturbing the water
[smpAttr:188]
@label=Moonlight
@helpText=Tick if there was bright moonlight
[smpAttr:189]
@label=Shoreline surveyed (%)
[smpAttr:190]
@label=Number of traps
<label for="observations">Observations:</label>
[species]
@id=observations
@rowInclusionCheck=alwaysFixed
@helpText=<b>Tick the boxes where presence is detected</b>. Then add numbers if possible in the boxes that appear.

Plants

=Plant Survey=
[date]
@helpText=Select the date of your survey
[location map]
@fillOpacity=1.0
@scroll_wheel_zoom=
@clickZoom=true
@helpText=Select your location from the drop down list or by clicking it on the map. If this is a new location it must be added to the list first.
<label for="observations">Observations:</label>
<p class="helpText helpTextAbove">
Check the box against species you observed and add % cover in numerals. Where % cover of a species was limited (e.g. if cover less than 1% of the pond <b>enter 0.1</b>.
<br>
To switch between common name order and scientific name order, update your <a href="/pondnet/user/me/edit/Preferences">profile preferences</a>
</p>
[species]
@id=observations
@columns=2

23

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Thanks Jim, that is exactly what I needed...will just help me iron out the odd item and bits of code.

Appreciated

Sam

24

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Jim/John,

Using the sample and occurrences form for an environmental data form (as I'm sure you said to do).  I get the following error when trying to submit data after set up:

The taxon must be specified. [occurrence:taxa_taxon_list_id]

I don't think this is the form structure I have set up that is wrong, but maybe the species section in the form set up.  As even though I don't have a species input control, because we don't want to add species using  this form, I have to select some of the options in the species setup....and I assume it is expecting an occurrence to be input.

How do I set up the form so that is error does not occur?

Many thanks

Sam

25

Re: Defra Species Surveillence project - setting up Indicia for monitoring

Hi Sam
Have you got a species list selected, chosen the Field used to filter taxa, and entered a single species in the Taxon filter items (in the manner selected in Field used to filter taxa)?
Jim Bacon.