1

Re: Version 6.13 released

All,
I have now published the latest version of Recorder (V6.13) on the downloads page of the Recorder website (http://forums.nbn.org.uk/recorderWebsite/upgrades.htm). This is a pretty major update to the package. I have also separately published an additional small update to the species dictionary (so the latest script is now 0000000E.sql) which is needed to make one aspect of the functionality work as expected. The update is available from the dictionary upgrades page (http://forums.nbn.org.uk/recorderWebsit … grades.htm)
Best wishes
Steve

2

Re: Version 6.13 released

Thanks for posting the update Steve. Does it matter which way around the species dictionary is updateed? i.e. Should the latest dictionary update be applied after the recorder update, or does it not matter? 

Cheers

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

3

Re: Version 6.13 released

It doesn't matter - you can apply the dictionary update even if you aren't upgrading the main software.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

4

Re: Version 6.13 released

Under contract to JNCC, I have updated the Help for version 6.13 of Recorder 6. Most of the new facilities are documented in more detail in the Help than in the release notes, particularly in relation to how to use them. The Home page of the Help has a section headed ‘Significant changes in Recorder 6 version 6.13’. This lists the new facilities with appropriate links to sections of the Help that provide more information. I hope this will assist users with getting the best out of the new functionality.

The revised Help can be accessed prior to upgrading your system by unzipping the download (Upgrade from V6.10 to 6.13.zip) then extracting Rec20Hlp.chm from the Help folder in UpgradeFiles.zip. Provided your system allows you to open .chm files you will be able open Rec20Hlp.chm and see the Help as it appears in Recorder 6.

Sally Rankin, JNCC Recorder Approved Expert
E-mail: s.rankin@btinternet.com
Telephone: 01491 578633
Mobile: 07941 207687

5

Re: Version 6.13 released

If we wanted to backup a server install before upgrading, what would we need to backup? Obviously there's the database and the Recorder 6 Server folder on the server, but is there anything else, like registry entries and suchlike?

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

6

Re: Version 6.13 released

I've now successfully installed onto our server. The only hiccup was with two of the .htm files (BiotopesMatch.htm and LocationsMatch.htm) in the Help directory - I was getting a permissions error on them. I deleted them from the server and the install then ran fine with the two deleted files getting replaced.

I suspect the permissions error may have been to virus scanner preventing html files from being modified.

Anyway, overall a fairly painless upgrade.

I was reading through the new features Sally mentions that are found in the help file and it struck me that people would probably want to read about them before downloading, so I'm pasting them in here. Note that the formatting and hyperlinks that take you further into the help file have been lost in pasting them in here, so do have a look at the massively improved help file instead if you are able):

Significant changes in Recorder 6 version 6.13
Survey tags - surveys can be organised into folders in the observation hierarchy using tags. This facility becomes available when Organise Surveys by Tag is ticked on the General tab in Tools - Options. Survey Tags can be used to determine the surveys included in reports, and those included in exports.

The way the system handles the Verification of identifications has been the subject of substantial revision. There are new settings for the verified flag on taxon and biotope occurrences:

Not verified


Failed/pending verification

Passed verification

This flag is set according to the determination type of the preferred taxon/biotope determination.

A Restrict Searches to box is now available on the Find Taxon dialogue when searching the taxon dictionary itself. If you want to do a wider search than just the Current Checklist, this setting can be changed, e.g. to Preferred Checklists, as elsewhere in the system. If a match from another list is selected, the other list will open with the selected item highlighted.

If 'Enable partial name search for taxa' is ticked on the General tab in Tools - Options, as well as searching using 5-letter abbreviations, you can search for taxa using parts of the name, separated by a space, e.g. in the VASCULAR PLANTS list "camp rotun" will find Campanula rotundifolia, Harebell.

When matching species in the Import wizard there is an option to search against All preferred lists.

Drag and drop can be used in the matching pages of the Import wizard for matching dictionary items (taxa and biotopes) and items from the main windows (observers, locations and documents).

Files to be imported by the Import wizard can now contain a Location Name column as well as Location and Grid Reference columns. Note that other column types are to be made available in this version but they were not available at the time of updating the Help.

The problems with the generation of keys when using the Import wizard to import data from MapMate have been fixed. The generation of the keys is now in line with the approach used in Recorder 2002 which means that if observations are reimported, e.g. because they have been updated, then the new version will overwrite the existing version provided they are always imported with a column in the import file containing the MapMate Key - see Import Wizard - data format.

To help avoid the problems with invalid items, when editing dates and location data in a survey event, you will be offered the opportunity to also change the same fields in its samples. When editing these fields in a sample you can change the same fields in its survey event and other samples in the same survey event. In the case of dates the change can be cascaded through the determinations.

Another facility to help avoid the problems with invalid items is the ability to revalidate data. System managers can revalidate items in the observation hierarchy and the other main windows, the results of the report wizard and the whole database. Also when exporting data it is revalidated. When revalidating data users have the option to view the results as a filter on the relevant window(s). These filters can be saved and re-loaded later.

When importing data:

the details of any invalid items can be saved to a file.


the imported data can be viewed as a filter on the relevant window(s) which can be saved and re-loaded later.

If measurement columns of the type Abundance of Qualifier (Unit), e.g. Abundance of None (DAFOR), are used on a recording card, the values entered can be restricted to a set of allowed values by specifying the allowed values on the measurement unit in the term lists. If a taxon occurrence data column is used in the Import Wizard its contents will be validated against the allowed values for the unit selected, if present.

If a species is no longer required on a recording card, you can remove it while using the card by right-clicking on it and selecting Remove Row from the pop-up box that appears.

If a delimiter, e.g. #, is entered for Rapid Entry Delimiter on the General tab in Tools - Options, data for all columns on a recording card can be entered as a delimiter separated string in the Add Species box, e.g. Blackbird#2#3 <Enter> will tick Blackbird, place 2 in the column after the name and 3 in the next column.

When entering taxa to a rucksack you can also add a Search Code. This code can be used in a Taxon/Search Text box when finding taxa and in the Add Species box when entering data via a recording card.

If when using the report wizard, you re-order columns in the Filter Result before saving the report, the change is remembered in the saved report.

Multiple locations can be selected for moving to a new position in the location hierarchy using <Shift> + left click of the mouse button or <Ctrl> + left click, as in Windows Explorer. They can then be dragged to the new position or moved to it by right clicking and selecting Promote to Top-Level or Move to. If Move to is used you are asked to select a location to move them to.

In a date field, pressing <F11> puts in the current date. In an editable field which requires an entry from the Names and Addresses window, pressing <F11> puts in the name of the user currently logged on.

If Display grid references as squares on map on the Spatial References tab in Tools - Options from the main menu is ticked, the distribution points on the map will appear as squares covering the area represented by the grid reference, rather than dots.

The option to Include or Exclude overlapping grid squares in report results has been added to the Polygon Selection page of the Report Wizard. Previously, an observation was selected for the report if its spatial reference fell within one of the selected polygons. Grid references like 10km squares, 1km squares or tetrads (2km squares) are represented internally by the spatial reference of the bottom left-hand corner. So, an observation for a 1km square, for example, was considered for the report if its bottom left-hand corner was within the polygon and irrespective of whether any other part of the 1km square was within the polygon. Now you have the option to specify whether you want grid squares that overlap the polygon boundary included or excluded from the report results.

Distribution points, the current polygon layer and all polygon layers can be exported to SHP files. Open the Map window and select Map from the main menu to access these facilities.

Export to Google Earth is a Report Output option on the filter result in the Report Wizard which allows you to save the data in a format suitable for use with Google Earth.

Export to ESRI Shape File is a Report Output option on the filter result in the Report Wizard which allows you to save the data in a format suitable for use with GIS systems like ArcView.

XML reports have been extended:

Rucksacks can be used in XML reports.


Polygons can be used in XML reports.

Note that the facility to use XML reports to update the database has been replaced by a new facility, XML Batch Updates, and it is no longer possible to use an XML report to update the database unless it is moved to the location for batch updates.

XML Batch updates can now be developed and run by users but they can only be run by system managers. Like XML reports they are text files with an extension of .xml and they are developed in a similar manner. By default, they are stored in the Batch Updates folder in the User Files folder in your Recorder system, unless the location has been changed using Tools - Options - File Locations (topic not updated). Select Tools - Batch Updates from the main menu to run them.

NOTE: as with anything that changes a database, batch updates should be thoroughly tested prior to running them on a live system.

Many new XML reports and a selection of batch updates are supplied with version 6.13 of Recorder.

Introductory documentation on writing XML reports and batch updates is available in the Documentation Wiki on the Recorder Software web site www.recordersoftware.org. The links to follow are Recorder 6 - Reporting - XML reports.

If you need assistance with writing XML reports or batch updates over and above that provided by the existing documentation, please contact the supplier of your copy of Recorder, or any of the Recorder suppliers who are listed on the Recorder Software web site. You can also share expertise or seek help from other users via the Recorder Forum, part of the NBN Forum, which can also be accessed from this web site.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

7

Re: Version 6.13 released

Thanks for that Charles. Back to your original question about backups - in order to backup the server install the only things you need are the database and the server folder. There are no registry settings on the server. Obviously each workstation also has it's own file folder and set of registry settings though.

Best wishes

John van Breda
Biodiverse IT

8

Re: Version 6.13 released

Hi

Thanks again, Charles, for helpful comments to get the Dictionary upgrade completed (from the How to thread). However, having everything in place, I think I have found a major glitch in the upgrade.

I needed to check a report and found that in the Report Wizard, the Taxon Measurements part no longer has children to select from. This is a major problem for creating reports. I didn't notice that these options about age classes, activity and abundance appeared anywhere else.

If they are missing can someone please do an emergency fix?

Cheers, Ian

9 (edited by johnvanbreda 04-07-2008 08:02:12)

Re: Version 6.13 released

Hi Ian

Thanks for pointing this issue out. If you are able to run the following script against your NBNData database it will fix this issue for you:

UPDATE Measurement_Context SET Context_Name='Taxon' WHERE Context_Name='Taxon Occurrence'
UPDATE Measurement_Context SET Context_Name='Biotope' WHERE Context_Name='Biotope Occurrence'

Best Wishes

John van Breda
Biodiverse IT

10

Re: Version 6.13 released

Hi John

Please forgive my ignorance, but I don't know what you intend me to do as I don't understand "run the following script against...".

Please say that in simpler terms!

Thanks, Ian

11

Re: Version 6.13 released

Hi

I've been away for a few days and it is a week since I posted the message above. "Run script", "script" are terms that do not feature in R6 help. Can anyone tell me what to do with the two lines of code John supplied?

Cheers, Ian

12

Re: Version 6.13 released

Hi Ian

Sorry, I should have explained. I've uploaded a Batch Update file which will fix it for you:
http://forums.nbn.org.uk/uploads.php?fi … ontext.xml

Here are the steps you need to follow:
1) Follow this link and save the file in your User Files\Batch Updates folder (within the Recorder 6 installation folder)
2) Restart Recorder (or start it if not already running), logging in with a user that has System Manager rights.
3) Select the menu option Tools\Run Batch Updates...
4) In the Select File box, choose Fix Measurement Contexts.
5) Click the Ok button. You will be prompted to backup your database - unless you have a recent backup this is probably a good idea although the script is very simple.
6) Once finished you will get a message "The batch update has been applied".

It should be fixed now. Finally it's probably a good idea to remove the Fix Measurement Context.xml file from the Batch Updates folder.

Let us know if you have any problems,

Kind Regards

John van Breda
Biodiverse IT

13

Re: Version 6.13 released

The spaces in the filename were being translated into %20 (the XML code for a space) on download which meant Recorder couldn't parse the file. Essentially, if you downloaded John's file and it causes an error when you start Recorder, please re-download it (I took the liberty of quickly fixing it).

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

14

Re: Version 6.13 released

Hi John and Charles

I have managed to get to stage 3 of the instructions above. When I open the Run Batch Updates and click on Updates I see "Pref1 Corrects Duplicate Preferred Determinations".

To try to change this I moved all the existing updates to a new folder so that the .xml file that I downloaded is the only one in the Updates folder - but I see no reference to it, and still get the above message, so cannot proceed further.

Am I meant to do something with the .xml file before doing the bit within R6?

Any suggestions?

I look forward to hearing from you,

Cheers, Ian

15

Re: Version 6.13 released

Hi John and Charles

I had another go and saw the correct message and progressed to stage 5 and clicked okay (I had previously done a back up). I got an Unhandled Error message:

Exception occurred in application Recorder 6 at 16/07/2008 20:45:49.
Version : 6.13.2.176

Exception path:
EOleException : Cannot open backup device 'NBNData_Backup'. Device error or device off-line. See the SQL Server error log for more details

Last event\actions:
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked
  actRunBatchUpdates invoked

Operating System : Windows XP  5.01.2600  Service Pack 2
Physical Memory available : 2,095,196 KB

DLLs loaded:
  advapi32.dll (5.1.2600.2180)
  comctl32.dll (5.82.2900.2982)
  comdlg32.dll (6.0.2900.2180)
  gdi32.dll (5.1.2600.3316)
  gdiplus.dll (5.1.3102.2180)
  HHCtrl.ocx (5.2.3790.2847)
  kernel32.dll (5.1.2600.3119)
  mpr.dll (5.1.2600.2180)
  MS5.Dll (5.0.0.12)
  MS5User.Dll (5.0.0.4)
  odbc32.dll (3.525.1117.0)
  ole32.dll (5.1.2600.2726)
  oleaut32.dll (5.1.2600.3266)
  shell32.dll (6.0.2900.3241)
  user32.dll (5.1.2600.3099)
  version.dll (5.1.2600.2180)
  winmm.dll (5.1.2600.2180)
  winspool.drv (5.1.2600.2180)


Any idea what is needed to make it work properly?

Thanks, Ian

16

Re: Version 6.13 released

Hi John and Charles

At last, it has worked. Only when the No option is clicked for not doing a backup. I do now have the measurement options in the report. Sorry if I find it a struggle to get R6 to do what you seem to find so simple!

All the best, Ian

17

Re: Version 6.13 released

Hi

Just before this version was released there were some exchanges about being able to:

- check for duplicate records already in the database, and
- on import check that the records-to-be-imported did not already exist in the database.

Has either of these been implemented and where do we find them? I have not been able to track them down.

Cheers, Ian

18

Re: Version 6.13 released

Hi Ian,

The Import Check did not get implemented, probably because the concept of a duplicate is not clear and it would have a performance impact on the rest of the import code. However, there is a report that should help now. Go to Reports\Run... from the menu. The report can be found under JNCC\Housekeeping report\H1 - Duplicate Records.

Let us know how you get on,

Best Wishes

John van Breda
Biodiverse IT

19

Re: Version 6.13 released

Hi Ian,

The data which can be imported can be extremely variable in content so it would be virtually impossible to come up with a  way of identifying duplicates which would satify all situations.  The report which is there attempt to identify duplicates based on a fairly restricted set of criteria. You could export the ouput from this into an Excel file, work out which ones you considered true duplicates and mark them.  The excel file could then be used (as a  CSV file) in an XML Batch update to delete the unwanted records automatically. With regard to imported records. You can now save details of the records imported, so you could write  an XML  report on these looking for duplicates based on the criteria you choose. The ones you don't want could then be deleted with an xml batch update.


Best wishes

Mike Weideli

20

Re: Version 6.13 released

Hi John and Mike

Thanks for indicating where the reports are. I tried them, once without specifiying a survey and another time with a particular survey. Both times the report did not work. With the first, I saw the hourglass for 30 minutes then checked with Ctrl, Alt Del and found that R6 was 'not responding'. The same occurred with the second try.

Does the report have to be run under specific conditions in order to work? If so, what are they? Any suggestions would be welcome.

Cheers, Ian

21

Re: Version 6.13 released

Hi

I have done further experiments and I can get several of the reports listed, including those in JNCC, Housekeeping to work and have seen their results. However, I cannot find a way to get either of the Duplicates reports to work. As soon as they start Recorder goes into a 'not responding' mode and even long waits means that nothing happens (I had wondered if, like the Dictionary upgrade, it was necessary to allow R to progress, even with a 'not responding' message). If it is me doing something wrong I am not sure what it is.

Am I the only person getting this response from Recorder or are others having the 'not responding' and no report produced?

Cheers, Ian

22

Re: Version 6.13 released

Ian

My guess is that we are just asking it to do too much. All the reports just produce an hour glass while they are running with no other message. Not responding doesn't necesarily mean it isn't working.  This one is trying to work on the whole database. It has been tested so I think it is working, but I will check to make sure and try and find a way of  dealing with the large amounts of data. One advantage of XML reports is that once they are fixed the new report can quickly be distributed without the need for an upgrade.

Try and run it as a quick report from a Survey.

Mike Weideli

23

Re: Version 6.13 released

Hi Mike

Many thanks for your suggestion. I have tried it on a survey with just 2970 records and the report did complete and took between 20 and 25 minutes. On that basis the check of my main survey of 82,000+ records will take at least 9 hours. I will probably set that up for this evening and see what transpires in the morning.

Good old La Fontaine again - "Patience et longeur de temps font plus que force ni que rage"!

All the best, Ian

24

Re: Version 6.13 released

Hi

I have run the duplicate report H1 and printed the results to check. However, I do not understand the date that is printed - it is not the event date, so I am finding it incredibly difficult, even in a survey with only 2970 records to identify the sample it is referring to. In the Observations list the records are shown by:

Event date, Event location

Without the event date how can we find the sample wanted? I see that there are Taxon_Occurrence_Key and Sample_Key printed, but cannot find a way of using either of them to identify the sample I need to check in the Observation list.

Even if I filter to sort out the records from a particular location it is not easy to check what the report is indicating needs to be examined as it does not return the Event Date. What is the date that it shows? It is not today, so is the day that the records were imported?

Can the report be amended to show the Event Date?

Cheers, Ian

25

Re: Version 6.13 released

Ian

The code suggests that  the date is the Sample date in the Recorder vague date format, but from what you say it sounds as though this isn't displaying correctly or is not of much use.  You can use  the goto key addin to get to the records based either on the Sample or  Taxon Occurrence Key. I have failed to get an answer out of it overnight on a database with 360,000 records, but it didn't fail and looked as thought it was still running when I abandoned it after 12 hours.  I have been thinking  this moning how it could be speeded up and may l have a go a doing something later today.   If I do I will add in the Survey Event date.

Mike Weideli