26

Re: R6 v6.18 candidates for inclusion

344 - Import wizard improvements please, particularly the ability to import using keys - taxon (TVKs?), location, individual - and an external reference.

Yes I agree with Teresa on this. Importing on TVKs would be good. It would certainly reduce time spent monkeying around with taxon matching for some of our imports.

Rich

Richard Burkmar
Biodiversity Project Officer
Field Studies Council

27

Re: R6 v6.18 candidates for inclusion

344 - Import wizard improvements please, particularly the ability to import using keys - taxon (TVKs?), location, individual - and an external reference.

I know I would say this, but I am in total agreement with Teresa on this point. So much time is spent by individual naturalists, schemes, societies and LRC's getting data into the right format. Why do we then ignore all this hard work within the import wizard?

I know there is a mechanism to export in the Rec NBN data format, but when I have imported through this means I have ruined our locations hierarchy and our individual table by entering masses of duplicate entries. Perhaps it is my lack of technical ability, but I cannot seem to be able to stop the duplicated site/individual info from entering the system (duplicate as in both have separate keys, but are the same person or site).

It would also help to have the ability to review and potentially discard keys that the import can't find within the system (preventing the potential for duplicate entries).

Eric

Eric Fletcher
RECORD Manager
http:/www.record-lrc.co.uk

28

Re: R6 v6.18 candidates for inclusion

From a personal point of view I must say how disappointed I am to see that this thread suggests that data is regularly being exchange between Recorder users using spread sheets. I always felt that the principle advantage of Recorder was that it allowed data exchange without the duplication of data and with custodianship being retained in one system. I always knew that getting this working would be a long term and uphill struggle, but one I thought worth aiming for in the long term.   It is more challenging  importing data in Recorder format and  having  to work with other users structure/locations etc. However, I still feel that the original idea of 'submit once - distribute many times' is the right goal to pursue and that importing data and altering in some way to fit the importing system is wrong in principle.

The Import Wizard was designed as tool  for importing data from users who did not want the complexity of Recorder. The idea being that they should submit their data to one Recorder 6 system who would import it and then pass it on as an R6 Export  to those who needed it.

My own personal view is that  it would be better to provide the tools to manage imported data more effectively, rather than finding ways to circumvent the principles on which the system is founded.

Mike Weideli

29

Re: R6 v6.18 candidates for inclusion

Even if the use of the import wizard to exchange data did not exist, we do still need to be able to import using existing keys from other sources, particularly Rodis and other data entry portals (I'm guessing indicia will be able to be set up similarly).

Spreadsheet templates we give out to recorders - e.g. for recording bats or butterflies - could easily have the TVK added to speed up import, and we could tell individuals their individual key etc.

Whether intended or not, the import wizard has become one of the most important and most used facilities in Recorder so improvements to it are really important to the users.

Apologies for muddying the waters. There is undoubtedly a discussion to be had/research to be done on why so many using Recorder day to day are not using it as it intended by the designers regarding transfer of information between Recorder systems, whether the original idea will ever be achievable, and how to persuade busy people to do things the difficult way rather than the easy way [the easy way being to keep comprehensive metadata]. However, I personally do not think that  failing to respond to current developments in online recording is a good way to make this happen - or that improvements to R6 data exchange should take priority over import wizard improvements.

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

30

Re: R6 v6.18 candidates for inclusion

Teresa

Once we have the priorities for versions 6.18/6.19 sorted it will probaby be worth starting a new topic to get more views on this and to establish what  the issues are. There are already a number of  proposed modifications which address issues which arise from having multiple version of names/location in systems so clearly there are users adhering to the principles and attempting to find better ways of working with data imported via the  proper mechanisms.

My personal view remains that the concepts of data integrity and data ownership/custodianship on which Recorder 6 is built are not something which should be cast aside lightly.

I don't see the above affecting the specific modification you have requested to allow TVK's and other keys to be imported. As you say there seems to be lots of good reasons for having this facility. In fact for uses like your Butterfly recording it should be possible using the current system to  implement something  which uses  Taxon_List_Item keys instead on taxon names.

Mike Weideli

31 (edited by DavidChun 29-04-2012 21:23:54)

Re: R6 v6.18 candidates for inclusion

It seems to me that the controls which Recorder provides and the principles on which they are based are essential for the transfer of data between systems. Without this users who submit data to LRC's and national organisations would feel uncomfortable about how it may end up being modified and duplicated without their knowledge.

There are, however, occasions when I need to  import data for reporting or research purposes only (ie. it is never going to be altered or passed on to other users or to the NBN). In these cases it should be acceptable to import this as new data from any source - including R6.  It can be deleted when it is no longer needed or deleted/replaced if it needs updating. In these cases the imported data may only need to be a sub-set or even a consolidation of the original information.  Having the ability to import using Taxon_List_Item of Taxon_Version keys would  make the import much easier and I can't see that it does any harm.  I currently identify such data at Survey level through a Survey type and allocate it all to a specific tag. While this is fine as long as I continue to remember to exclude it from exports, there is the possibility that I might forget or that the importance of the Survey type is not recognized at some stage in the future when the data is passed over to someone else.  Would it  be possible to have a special Survey Type whcih is always excluded from exports ?

32

Re: R6 v6.18 candidates for inclusion

Report Wizard requests (I know these have been made before but to reiterate)
- export to .xlsx
- Grid reference qualifier
- Family

I like Alison's idea of a separate field for data protection sensitive location info i.e. house numbers/names

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

33

Re: R6 v6.18 candidates for inclusion

MikeWeideli wrote:

From a personal point of view I must say how disappointed I am to see that this thread suggests that data is regularly being exchange between Recorder users using spread sheets.

Mike, the primary reason I have heard time and time again for this is that accepting imports in Recorder format 'messes up' an otherwise tidy database. In other words, people like to keep their surveys, locations, individuals and the like nicely ordered. Imports from other copies of Recorder will pollute this order. Importing from spreadsheet means that the order can be controlled. This isn't just an aesthetic issue either: having a well structured locations hierarchy or surveys hierarchy can make reporting and exporting much simpler too.

So, we've identified a need, right? People are working around what they consider to be a problem. How can Recorder be 'fixed' so that data can flow from site to site in native format while allowing the site owner to retain control over the structure. Survey tags were a step in the right direction, but still aren't a solution I feel.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

34

Re: R6 v6.18 candidates for inclusion

I absolutely agree with Charles, particularly in relation to locations. This might not work but the thought just struck me that you could have different 'determinations' for location. E.g. Charles sends me some data but his location names differ from those in my system. I redetermine the locations to fit with my system and set as preferred but keep the imported locations in a separate structure.

Gordon Barker
Biological Survey Data Manager
National Trust

35

Re: R6 v6.18 candidates for inclusion

charliebarnes wrote:

In addition to these, we've also got some fixes/improvements to suggest (again in roughly this order):

* Ability to use the hierarchical tag structure in the observation window
* Allowing importing of “site” records and auto-assigning grid reference from location details
* Highlight failed/pending verification, confidential and zero count records in the observation hierarchy
* Sort Measurement Qualifier drop down alphabetically

We may be able to provide funding for some of our suggestions, and possibly 216 if needed. Obviously we would need to know how much these changes would cost - even a ball park figure would do. Is it possible to put some £££ next to some of these changes (ours or those on the spreadsheet - or even past changes) ?

Just wondering if there has been any progress on this?

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

36

Re: R6 v6.18 candidates for inclusion

We are working on v6.18 at the moment – testing of the changes is underway – but once that is out I am hoping we will be able to look again at the outstanding suggested enhancements with a view to costing some of them. The main focus for v6.19 will need to be the changes for the revised taxon dictionary but we will be considering other enhancements.

Sally Rankin, JNCC Recorder Approved Expert
E-mail: s.rankin@btinternet.com
Telephone: 01491 578633
Mobile: 07941 207687

37

Re: R6 v6.18 candidates for inclusion

Sally,

I'm now running 6.18.1 and as usual, I've found something I would like which isn't in the latest version:

In reporting under Additional Filters, the ability to filter on Sample Method, I can't see this field being an option in the list

Craig

Craig Slawson
Staffordshire Ecological Record

38

Re: R6 v6.18 candidates for inclusion

This looks like it is partly implemented, but that there is something wrong with the entries in the relevant tables. If so we should be able to provide a speedy fix.

Mike Weideli

39

Re: R6 v6.18 candidates for inclusion

The attached SQL should fix the problem with Sample Type on the filter. Also noticed that Obs Surveyors Reference wasn't set up correctly and have fixed that.

Post's attachments

FixSmapleTypeandObsReference.sql 801 b, 3 downloads since 2012-12-22 

You don't have the permssions to download the attachments of this post.
Mike Weideli

40

Re: R6 v6.18 candidates for inclusion

There is an error in the SQL Mike. It begins with the line USE NBDATA where it should say USE NBNDATA. Other than that it seems to work fine.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

41

Re: R6 v6.18 candidates for inclusion

Brilliant, thanks Mike and Rob - worked a treat!

Craig

Craig Slawson
Staffordshire Ecological Record