1 Reply by davec (edited by davec 22-01-2008 15:22:45)

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Multiple Surveys can be imported - bug or feature???

We had a dataset give to us in NBN Zipped format. When it was imported, we noticed lots of odd surveys had been added to our DB. When I looking into the unzipped MDB the SURVEY table has 28 rows!

Given that one can only export one survey at a time from Recorder (Am I right about that?) then someone has appended many Recorder exports into a single file using access. What I find amazing is that Recorder imported the file!

This means that in future we will take each zipped file apart first and have a look at it. (Even if it is from a trusted source!)

Any news on the delete survey add-in?

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

2 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: Multiple Surveys can be imported - bug or feature???

Dave, there are a number of ways to export data in Recorder which would include multiple surveys:

* If results from the report wizard are exported and span multiple surveys, then all of the surveys are exported
* If a location that is used in multiple surveys is exported and occurrences are included, then all of the surveys are exported.
* The Tools > Export Management tools (Contribute to Scheme and Export Filter) can export multiple surveys
* If a recorder who appears in multiple surveys is exported and all observations are included, then all of the surveys are exported.

There are possibly others, but you get the idea. :)

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

3 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: Multiple Surveys can be imported - bug or feature???

Another point worth mentioning is that as long as it is valid data, then Recorder will import just about anything via an mdb. So you could import just one table if you wanted.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

4 Reply by davec

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Multiple Surveys can be imported - bug or feature???

Charles, thanks for the hat tip on the ways multiple surveys can get exported. This is going to add another layer of data management now to our processes as the vast majority of the surveys that were in the file were not needed. I guess this begs the question that some sort of data viewer be built into Recorder to check what one is about to import. We have used access sometimes to check a few zipped MDBs in the past, but not all of our data workers have this.

Also a feature to select just those surveys what one wants to import would be needed too.

What makes the file suspect is that normally the tables names within the MDB are uppercase, I noted that a few were lowercase. This led me to think they had been manually edited.

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

5 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Multiple Surveys can be imported - bug or feature???

Hi Dave

Charles is absolutely correct as usual, there is nothing stopping you exporting multiple surveys at a time, for example using an export filter or report. The issue you mention goes much further than surveys though, essentially the import file is a black box (unless you rummage in it yourself by unzipping it and using Access) and you are not aware what additional records have been imported. The good news is that the next version of Recorder which we are busy developing at the moment contains a new facility allowing you to review the data that has just been imported. It should be ideal for situations like this.

Best Wishes

John van Breda
Biodiverse IT

johnvanbreda's Website

6 Reply by davec (edited by davec 23-01-2008 10:11:44)

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Multiple Surveys can be imported - bug or feature???

Hi John,

Thanks for the info. What you mention sounds like a very welcome addition to Recorder's data management. Would it be possible to preview the data before import, and select which of the surveys one needs. Also, a simple text file (CSV would be great) logging what data has gone where would be valuable, thus allowing such metadata to be used in data managment.

Kind Regards

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

7 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Multiple Surveys can be imported - bug or feature???

Hi Dave

It's not that easy to allow you to preview the data before import, simply because the data sources are so heterogenous and are not in Recorder's SQL Server database format until they are actually in the database. However you will be glad to know that the new Import Wizard will allow you to save a tab-separated text file listing the records changed by the import, which you can either review immediately or load into Recorder as a filter to review the changes later.

Regards

John van Breda
Biodiverse IT

johnvanbreda's Website

8 Reply by davec (edited by davec 24-01-2008 10:21:34)

  • davec
  • Veteran
  • Offline
  • From: Brecon, Powys, Wales
  • Registered: 11-05-2006
  • Posts: 108

Re: Multiple Surveys can be imported - bug or feature???

Hi John,

The tab sep. file is excellent news! :)

It's a pity about a data preview. I haven't seen all permutations of import data in the MBD files, but the ones that I have used  all had tables which seem to be a subset of the NBN model (SURVEY, SURVEY_EVENT, SAMPLE ... etc ). I can see that when this is not always the case how that would make life difficult. As a thought, what about the temp DB that the import wizard creates. If that is used to form the model could that be a good point to preview?

Maybe preview would not be such a big issue for me if we had a delete survey or delete by filter mechanism.

Cheers.

Dave Cope,
Biodiversity Technology Officer,
Biodiversity Information Service for Powys and Brecon Beacons National Park.

davec's Website

9 Reply by Sarah Shaw

  • Sarah Shaw
  • Veteran
  • Offline
  • Registered: 23-06-2006
  • Posts: 267

Re: Multiple Surveys can be imported - bug or feature???

Hi Dave

There is a Delete Survey addin on it's way - it's something we have been promising for a while.

It's actually been developed (was finished on the last day before the Christmas holidays) (by Stuart Ball) but requires some testing before we release it.

I'm hoping to be able to do this next week and so long as I don't come across any problems I can make available to users then.

I'll let you know how I get on.

Best wishes,

Sarah Shaw
Biodiversity Information Assistant
JNCC