1

Topic: Inconsistent Total Records

Hello all,

I am trying to generate some figures regarding the total number of observations in our database, and how many have been entered over differing periods of time.  Problem is that the standard reports seem to be giving different totals so I cannot trust the figures.  There is probably a perfectly reasonable explanation that I am missing?

22090 = Record Count V06-12
13642 = Count of records entered within <date range> V07-12(since 01/01/1800)
24862 = RecordStats

Thanks in advance...

Mike Beard
Natural Course Project Officer
Greater Manchester Local Records Centre

2

Re: Inconsistent Total Records

Does Record Count V06-12 give you the same number as on the Welcome screen (Help – Summary Info)? For some reason Count of records entered within <date range> V07-12 is excluding observations with date Unknown but that would seem to indicate that there are a lot of date Unknown observations in your system. Are you running it from System Reports\Statistics? There are some old versions of various reports that appear elsewhere in the hierarchy unless they have been cleared out. It is hard to say what is going wrong without looking at your database. RecordStats is counting taxon determinations rather than taxon occurrences, hence the higher figure.

Sally Rankin, JNCC Recorder Approved Expert
E-mail: s.rankin@btinternet.com
Telephone: 01491 578633
Mobile: 07941 207687

3

Re: Inconsistent Total Records

Hi Sally, thanks for getting involved.

Record Count V06-12 gives the same result at the Summary Info (22090).

It does not seem to matter which of the two menu options Count of records entered within <date range> V07-12 is being run from:
System Reports\Statistics
Statistics
In fact the XML looks to be identical apart from the MenuPath.

It makes sense for a date range query to exclude unknown dates.  Guided by the XML I checked Survey_Event.Vague_Date_Type and did not find any with 'U'.  I was looking in an exported Access database as that is the only way I have to see things at a table level.  A handful of records were excluded from the export for validation errors, but not on the scale being seen in the total figures and no mention of things being excluded for unknown dates.  Taxon_Occurence has 22084 records, which seems reasonable if you take into account the validation errors.

RecordStats claims to be counting records by year added, a bit misleading?

Is it worth seeing if other people are getting strange results (surely the issue would have come up by now?)

Maybe I should send the export file to you?

Mike Beard
Natural Course Project Officer
Greater Manchester Local Records Centre

4

Re: Inconsistent Total Records

My measurements:

1980750 = Record Count V06-12 (this is just a count of occurences)
1979081 = Count of records entered within <date range> V07-12(since 01/01/1800) (but if I remove the restriction on "unknown" dates I get 1980750)
2185132 = RecordStats (this report is working off taxon determinations, rather than occurences hence the increase)

1980750 = Summary Info dialog box

So my figures seem to sum up okay.

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

5

Re: Inconsistent Total Records

Thanks Charlie.  Looks to be my data then.

Mike Beard
Natural Course Project Officer
Greater Manchester Local Records Centre

6

Re: Inconsistent Total Records

Just to be thorough I followed Charlie's route of removing the "unknown" dates restriction in the XML, but it makes no difference.

Mike Beard
Natural Course Project Officer
Greater Manchester Local Records Centre

7

Re: Inconsistent Total Records

I have taken a look at the export file you sent me and have found that the Entered_By field on a lot of taxon occurrences contains a name key that isn’t in the Name table or the Individual table. The same applies to survey events. Count of records entered within <date range> V07-12 only includes taxon occurrences when the Entered_By on the survey event has a corresponding entry in the Individual table so this is the reason for it returning such a low count. The Recorder 6 user interface should not allow these problems to arise but they could have happened as a result of system crashes, power cuts, bugs in earlier versions of the system or changes done incorrectly by users with Microsoft Access or SQL Server in the database itself. As the problem names are from other systems you may be able to rectify things by reimporting them, assuming they aren’t missing in those systems. This can be done by creating a rucksack containing the names, opening it and using Tools – Export Data to do an export. With a rucksack open it will export the things in the rucksack and give you the option of whether to include the associated observations.

Anyone facing issues like these would be well advised to use the package called Problems V2.zip to check for various problems. It can be downloaded from http://forums.nbn.org.uk/viewforum.php?id=33&p=5 . It contains a number of xml reports and batch updates designed to detect and correct problems in Recorder 6 databases. The documentation in it has been written to help users with correcting the problems found. Running System Reports – Information – Sy06 - Problems(Summary) via Reports – Run will provide a summary of the problems. This often reports thousands of problems many of which can be corrected by batch updates. Running the batch update DATA_Correction 1 will correct several types of errors very quickly. If the reports and batch updates aren’t already in your system the documentation in Problems V2.zip explains how to install them.

Sally Rankin, JNCC Recorder Approved Expert
E-mail: s.rankin@btinternet.com
Telephone: 01491 578633
Mobile: 07941 207687

8

Re: Inconsistent Total Records

Thank you for your help Sally, especially when you had other things to be getting on with.

Just waiting for the missing Entered_By names to be sent to us.  I will provide an update when we have received and imported them.

Mike Beard
Natural Course Project Officer
Greater Manchester Local Records Centre

9

Re: Inconsistent Total Records

We have now imported the missing names and the totals match up so thank you to everyone involved.

Mike Beard
Natural Course Project Officer
Greater Manchester Local Records Centre