1 Reply by Sarah Shaw (edited by Sarah Shaw 20-04-2007 10:36:17)

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Jude

As far as I'm aware this should work as it was tested by one of the resellers. We did have 1 problem in the past with this, but this was with a standalone copy on a network.

John - can you confirm that this is possible and under what circumstances this option would be greyed out?

Additional note - could this be permissions related? When I log on to my test computer (no copy of Recorder 2002 present on the machine) as just 'Recorder User' this option isn't available, but with an admin account it is?

Any help much appreciated,

Sarah

Sarah Shaw
Biodiversity Information Assistant
JNCC

Sarah Shaw
Biodiversity Information Assistant
JNCC

2 Reply by Sarah Shaw

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Also for general information - it is worth reading the following warning displayed before you transfer Recorder 2002 records:

Transfering Your Data

You are about to transfer your records from your old copy of Recorder 2002 into your copy of Recorder 6. This transfer process will completely overwrite the current database and any data that may have been transferred or input previously. If this fails for any reason, you can repeat the exercise. Once you are happy that your data has transferred properly, you should uninstall Recorder 2002 using this option provided on the CD and start to use Recorder 6 as your master database.

The transfer requires about 900MB in your database directory for every 500,000 species occurrences.

WARNING

Since your new copy of Recorder 6 and the old copy of Recorder 2002 have the same SiteID, any records you enter into Recorder 6 will have the same codes as any you enter into Recorder 2002. This means that the records will overwrite each other and your data will be lost. DO NOT enter new data into Recorder 6 until after you have removed Recorder 2002 from this machine. DO NOT allow your copy of Recorder 2002 to be used by somebody else.

Best Wishes,

Sarah

Sarah Shaw
Biodiversity Information Assistant
JNCC

Sarah Shaw
Biodiversity Information Assistant
JNCC

3 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

If you are using the Standalone Recorder 6 CD then this option will grey out if Recorder 2002 is not installed on the machine. However, the network install CD for Recorder 6 should let you do this by copying the nbndata files across from somewhere else as you suggest.

John van Breda
Biodiverse IT

johnvanbreda's Website

4 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Jude,

Thinking this through, I think the problem is because the migration of data from Recorder 2002 to Recorder 6 needs to be done on the database server. The SQL Server cannot see your local hard disk so it cannot migrate Recorder 2002 data from there, it has to happen from the database server machine itself.

Regards

John van Breda
Biodiverse IT

johnvanbreda's Website

5 Reply by Sarah Shaw

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Jude and John

I have just realised that you may not need the CD after all? In the Recorder 6 folder on your computer you should be able to see the following file 'WorkstationSetUp.exe'. If I double click on this from my test workstation (logged in as an administrator) the welcome screen is displayed and I am given the option to 'Copy data from Recorder 2002 into this copy of Recorder'. However, I haven't tested beyond this.

John - can you confirm that using this approach is ok?

Many thanks,

Sarah

Sarah Shaw
Biodiversity Information Assistant
JNCC

Sarah Shaw
Biodiversity Information Assistant
JNCC

6 Reply by Sarah Shaw

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Jude

Just something work double checking - has the source folder got all of the Recorder 2002 database files in there? This includes:

Index_Taxon_Group.mdb

Index_Taxon_Synonym.mdb

Index_Taxon_Name.mdb

nbndict.mdb

I also have the following files in my Recorder 2002 database folder - UpgradeErrorsfirstrun.mdb and UpgradeErrorsfssecondrun.mdb, though not sure if these are critical for the transfer or not.

Best wishes,

Sarah

Sarah Shaw
Biodiversity Information Assistant
JNCC

Sarah Shaw
Biodiversity Information Assistant
JNCC

7 Reply by davec (edited by davec 21-06-2007 08:53:35)

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Sarah Shaw wrote:

I have just realised that you may not need the CD after all? In the Recorder 6 folder on your computer you should be able to see the following file 'WorkstationSetUp.exe'. If I double click on this from my test workstation (logged in as an administrator) the welcome screen is displayed and I am given the option to 'Copy data from Recorder 2002 into this copy of Recorder'. However, I haven't tested beyond this.

I can confirm this worked (well, the transfer is 99.9% complete and no exceptions so far). I have managed to get to version 6.10.4.120 from a 6.9.3 install. I tried to transfer data across from Recorder 2002 using the transfer item in the installer (CD). This would not actually give me a transfer button and keep going around to the first page of the dialog.

I tried the suggestion given by Sarah to use the "WorkstationSetUp.exe" program and this worked. Given that the transfer process cannot use network drives, I had to copy all of my Recorder 2002 files over to a temp directory on the server. I noticed that not having  "UpgradeErrors.mdb" caused an error in the transfer. That file was in my Recorder 2002 database directory. I also needed the various index files from that directory too.

As I write this, the transfer (which has been running all night - my test server has little memory and a sub-GHz CPU) is "Finalising data migration" and the progress bar is just off the end point. I have no idea how much longer it will take. A console type display showing finer grained progress would be very helpful to determine if the transfer has hung.

If it succeeds, all I have to do now is work out why it has reported the transfer of 2,107,885 records from a Recorder 2002 database of just over 600,000 occurrences. Then dig through the 60,317 errors generated. Oh joy! (I do know our Recorder 2002 database is "unclean" caused from various import crashes and other oddities and it is on our TODO list to clean this up before we commit to Recorder 6).

Cheers.

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

8 Reply by brianmiller

  • brianmiller
  • Veteran
  • Offline
  • Registered: 10-05-2006
  • Posts: 113

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Dave,

Not sure how this will end for you but I tried migrating the data from R2K2 to 6.10 and came up with the error

Exception path:
EOleException : Invalid column name 'Full_Edit_Own_Data'

This was a problem reported by Graham Hawker for R2K2 to 6.9.3

http://forums.nbn.org.uk/viewtopic.php?id=309

I had the same problem (on a standalone PC) and the answer was to transfer the data on earlier version then do all the upgrades.

Hope yours works and you do not have to go through this one.

Brian

Brian Miller
(Conservation Officer (Buckinghamshire), BBOWT)

brianmiller's Website

9 Reply by davec

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Brian,

Thanks for the pointer on the migration.

Given that I had no feedback from the transfer dialog apart from the one mentioned in my post, I decided to log on from a workstation to check the summary dialog. When I looked back at the server, an ODBC error dialog had popped up. Dismissing that closed the transfer.

Comparing statistics, there is a loss of 32 species observations and a loss of 100 habitat observations from Recorders 2002 to 6.

I've just looked into the UpgradeErrors.mdb file and I guess I'll have to use that with the Recorder 2002 nbndata.mdb file to construct anything meaningful.

I've looked for a plain text log file of what the transfer was doing, but so far found nothing. It would be useful - is essential - to have human readable, easy to understand database logs to save us days of work trying to debug migrations.

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

10 Reply by davec (edited by davec 21-06-2007 10:51:18)

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Just quick follow up concerning the "UpgradeErrors.mdb" file.

Before the migration test this file was 11,350KB, timestamp for last access was 14/11/2005 13:50. After migration it was 11,508KB with a timestamp of 20/06/2007 19:09 (The migration was started on 20/06/2007). So it has added information, but there is nothing in the database to indicate what. (I can easily see that a TAXON_OCCURRENCE table with 191 rows in it has been created)

Does anyone know if this database is created from scratch on transfer? (My experience suggests that the transfer needed a pre-created file as given by a error dialog during transfer).

Is the existing data in UpgradeErrors.mdb deleted before transfer, thus using existing tables. (Seems unlikely to me - far easier to create a new error database from stratch).

Without metadata for this process and the knowledge of what processing took place with this file, it is effectively useless.

I will revert back, as suggested by Brian, to 6.9.3 and retry the transfer. If that process creates a new fresh UpgradeErrors.mdb then at least I will know the source of the data.
This issue with the error database could be a side effect of a migration attempt directly in 6.10 as indicated in the above post.


Cheers.

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

11 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

I completely agree that the various logs and error files generated by Recorder when doing imports and upgrades need some serious attention in order to make them human readable. Also, being able to perform validations on existing data is essential; the present situation where data are only validated on import means that invalid data could sit in the database for a long time before being discovered.

A further problem related to all of this, this time with the import wizard: if invalid items are found (for example, events that fall outside the date range of their survey), then a list of ID codes are presented right before the last step of the import process. Clicking on each of the codes gives a reason for the data being invalid, but offers no way of correcting it, or referencing back to the original data. I've been importing tables containing 90,000+ records and finding 50 or so invalid items is proving to be difficult. Just being able to dump the records in some human readable form would be of great help. I had a look for a temporary .mdb but couldn't find anything.

Charles

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

12 Reply by sallyrankin

  • sallyrankin
  • Administrator
  • Offline
  • Registered: 07-04-2006
  • Posts: 371

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

I haven’t seen any documentation to confirm whether or not it is possible to transfer data from Recorder 2002 directly to Recorder 6 v6.10 at the moment. Because the transfer process needs to load up an empty database (see my post of 22/02/07 - http://forums.nbn.org.uk/viewtopic.php?id=262) I had assumed it would be best to wait until the v6.10 installation CDs are available – due 29 June according to Sarah’s post 18/05/07 (http://forums.nbn.org.uk/viewtopic.php?id=361#p1643). In the post following mine, John van Breda effectively says that if you used a v6.7 CD to transfer data to Recorder 6 v6.9 you would then have to run the v6.9 upgrade again to upgrade your database but I haven’t tested this. Also, if you don’t use a CD to do the transfer I don’t know where the transfer process gets an empty database from.

Information on where to find the temporary database created by imports is in my post of 7/04/07 - http://forums.nbn.org.uk/viewtopic.php?id=324. There is more information in the training document prepared by Stuart Ball for the Falklands.

The issue of providing a mechanism for validating data in a Recorder database was raised at a meeting with JNCC several years ago. I thought at the time that they were agreeable to providing it but presumably the funding is/was not available. Has anyone any ideas on how it could be funded?

Sally Rankin, JNCC Recorder Approved Expert
E-mail: s.rankin@btinternet.com
Telephone: 01491 578633
Mobile: 07941 207687

13 Reply by Sarah Shaw

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi

I have begun to test the data transfer from R2002 to Recorder 6 with the latest Recorder 6 (v6.10) CD's (which has gone ok so far), but still need to do some further testing in this area. I'll let you know how I get on in a couple of days time.

I have also made a note of the improvements suggested here - I agree that the process for tracking down problems with data migration (from R2002 for example), data import and upgrading could be much improved.

Dependent upon the amount of bugs reported that we need to fix, we are hoping that there will be scope in the next version to develop and improve some of Recorder 6's functionality. Any changes that we do make will be carefully prioritised to ensure that those developed will be of the greatest benefit to users.

Kind regards,

Sarah

Sarah Shaw
Biodiversity Information Assistant
JNCC

Sarah Shaw
Biodiversity Information Assistant
JNCC

14 Reply by Ed Rowsell

  • Ed Rowsell
  • New member
  • Offline
  • From: Chichester Harbour
  • Registered: 30-10-2007
  • Posts: 2

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi,

Bit new to all this, but seem to have managed to set up R6.10.4 on an MSDE instance, on a new machine, our 2002 installation is on an old laptop.

Tried to use the transfer of data from R2002 function on the installer disc, initially it would not accept a copy of the database folder copied onto the drive, I installed R2002, whereafter it could "see" it however it resulted in a repeat circle failing to get to the transfer button. Attempted the transfer with the workstationsetup.exe as suggested above, and all seemed good, until it hit its first error whereupon it is reporting an error that it cannot find UpgradErrors.mdb, if I ok the error popup transfer stops. The problem is that the original installation does not have does not have this *.mdb so I am a bit stuck at the moment.

Cheers

Ed Rowsell
Chichester Harbour

15 Reply by Sarah Shaw

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi Ed

There are some potential pitfalls with the transfer - did you read the following guide

http://forums.nbn.org.uk/files/Guidance … 104_V3.zip ?

If you have and problems still prevail please let me know.

Best wishes,

Sarah Shaw
Biodiversity Information Assistant
JNCC

16 Reply by Ed Rowsell

  • Ed Rowsell
  • New member
  • Offline
  • From: Chichester Harbour
  • Registered: 30-10-2007
  • Posts: 2

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi,

Sorry didn't read right to the bottom of the document, the install had failed to copy over NBNdata.mdb into the R6 directory.

All seems to have transferred over now (we haven't go that many records in recorder to be honest)

Cheers

Ed

17 Reply by Ed Hudspeth

  • Ed Hudspeth
  • Member
  • Offline
  • Registered: 04-03-2008
  • Posts: 12

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Hi,

I've followed the instructions in the this guide http://forums.nbn.org.uk/files/Guidance … 104_V3.zip  and i get the error: Could not find file: UpgradeErrors.mdb (and is looking in the folder where i copied the R2002 database to), it only allows me to click ok, and then it returns to the Login Options screen, where i can again click transfer and the same happens again.

Ed

18 Reply by sallyrankin

  • sallyrankin
  • Administrator
  • Offline
  • Registered: 07-04-2006
  • Posts: 371

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

When you get this error simply create an empty Access database called UpgradeErrors.mdb in the Recorder 2002 Database folder. It doesn't seem to matter what version of Access you use. I don't know why the transfer facility doesn't create this in some circumstances but this gets around the problem. I create a new one each time I do the transfer as I have seen it simply append the errors to what is already there so you don't know which ones relate to which attempt to transfer the data.

Sally Rankin, JNCC Recorder Approved Expert
E-mail: s.rankin@btinternet.com
Telephone: 01491 578633
Mobile: 07941 207687

19 Reply by Ed Hudspeth

  • Ed Hudspeth
  • Member
  • Offline
  • Registered: 04-03-2008
  • Posts: 12

Re: Copy Data from Recorder2002 into Recorder 2006 (Network Version)

Thanks very much. I did this and it worked fine, data migration complete!