1

Topic: Recorder 6 forward plan: position statement

Recorder 6 forward plan: position statement

JNCC would like to comment on plans for the development of Recorder 6 on behalf of the Recorder 6 Steering Group (JNCC, Natural England, National Resources Wales and Department of Environment NI).

A number of online data management tools are under development for species records. There is a clear strategic interest in closely aligning and integrating these developments across the National Biodiversity Network (NBN). JNCC has acknowledged the significance of this emerging strategy, and the benefits of moving to an online data management system are currently being considered as a possible alternative to traditional standalone desktop software like Recorder 6.

Future decisions relating to online data management tools as part of the NBN will likely have implications for our future investment in Recorder 6. No decision has yet been taken and there are no plans for any change within the current financial year. JNCC would like to stress that the implications for Recorder 6 from any future plans that do emerge will be medium to long term.

If and when an integrated online data system is adopted within a plan for replacing Recorder, a full consultation will be run with stakeholders to ensure all views and concerns are considered and addressed.

Joint Nature Conservation Committee

24 September 2013
Contact: mary.campling @jncc.gov.uk

2

Re: Recorder 6 forward plan: position statement

I am sure I am not the only one to put a great deal of time and effort into developing Recorder as my main system for holding data. This has taken a number of years and it seems totally unreasonable that it looks like JNCC is considering reducing support for Recorder and its continued development. There is currently nothing available on line which has the flexibility to hold the data which I require and as this stand at the moment I can not see there being anything available in the near future. Even  assuming that I would ever want to put my data on some on line server controlled by someone else. 

David

3 (edited by RobLarge 26-09-2013 12:03:03)

Re: Recorder 6 forward plan: position statement

I agree totally David and I am also quite sure that we are not alone.

Let us hope that JNCC will actually listen to us when the time comes. I am not going to say much more now, because we are clearly at an early stage in the process. But I will have plenty to say in any future consultation.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

4

Re: Recorder 6 forward plan: position statement

I think this has been coming for a while when you consider how online and "cloud" stuff has been burgeoning in recent years. Considering the requirements of such a system and the time it will take to get something robust enough to be functional and easy enough to use I'd be surprised if anything was in place in the next 5 years.

Graham Hawker
Thames Valley Environmental Records Centre

5 (edited by nmrs 26-09-2013 16:02:39)

Re: Recorder 6 forward plan: position statement

David/Rob/Graham, You're not alone guys!

Les Evans-Hill
Senior Data Officer, National Moth Recording Scheme

6

Re: Recorder 6 forward plan: position statement

Like others I don't want to say much now. Just that I would be surprised if many current Recorder 6 users, institutional/NSS/LRC or personal, would agree that robust online data management, with all the required fields, importing and reporting functionality of their primary database is possible within a 5 year timeframe. I don't know how many would view it as desirable; I think I can safely say my LRC and museum would not and would have to start looking at alternative software solutions at the point R6 support ceased, such as those bespoke databases already used by some LRCs.

User Group?

Whilst I respect that JNCC own Recorder 6, I have thought for a while that there is a need for R6 users to have an independent user group which might be able to collate and input all our needs more collectively for any future development (if there is any). This forum is of course a great place to share daily problems and solutions, with the support of the R6 consortium paid for by JNCC which we do appreciate, although perhaps not vocally enough.

Would anyone else be interested in forming some sort of user group, perhaps with an email list and eventually a committee etc? Whilst stakeholders will still need to respond to any consultation separately, it might allow for more discussion and liasion with the Steering Group if they thought that would be useful?

Some dbms have such independent groups, for example there are country based Oracle user groups e.g http://www.ukoug.org/about-us/ which share customer needs with the Oracle company or indeed the SQL Server user group.

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

7

Re: Recorder 6 forward plan: position statement

Let's not get too far ahead of ourselves here. The fact that the Recorder Steering Group (which I'm not on and have no insight into) are considering ways for Recorder to evolve and remain a sustainable, viable platform is reason to feel highly encouraged. IT rapidly evolves, and the underlying codebase of Recorder is already aging and based on a programming language that is becoming obsolete and increasingly difficult and expensive to support with a pool of programmers that is diminishing. No matter how much we invest in current systems (and lord knows, we've invested enough in Recorder 6 here in Sussex), we must be able to face the future, adapt and embrace change. Or else we face extinction.

The clear trend in IT is away from the desktop to distributed wide area (aka 'cloud', Internet and Web-based) services. We resist that at our peril. But don't think the rise of the network means the destruction of the desktop (and increasingly native apps on tablets and smartphones) - it merely means a realignment and a new symbiosis.

There is also a strong trend away from proprietary software, with the UK being one of the many EU countries to mandate a preference for open source and open standards. There is also a further mandate to consider 'cloud' solutions before others (See also G-Cloud).

Further salient themes were picked up at the recent FOSS4G conference in Nottingham.

So let's not be too negative. Let's look at the requirements, the pros and cons of different approaches, and carefully consider where IT (for users and developers) is headed, ride with the wave rather than against it, and start a constructive dialogue on where we want to head, rather than be dragged unwillingly behind. We have the opportunity here to lead rather than be led.

And for what it's worth, I think Graham is right - 5 years seems a reasonable time-frame. Optimistic if anything.

Stop-press: Just before hitting post, I've seen Teresa's message. A user group is a great idea. There used to be one and I went to several meetings at JNCC in Peterborough. Not sure what became of it, but it seems like a very good idea to get a good cross-cutting representative group going again.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

8

Re: Recorder 6 forward plan: position statement

Agreed Charles, I have no particular objection to web-based systems, indeed I recognise many of the benefits they offer, such as platform-independence, ease of updating etc.). If that is the way Recorder goes in its next incarnation I have no problem with that.

I am less convinced however by the movement towards the cloud, which it seems to me offers more benefit to the providers than to the users. What do we gain apart from automated backups? And what do we lose by ceding ultimate control of our data apart from the freedom from service contracts? I am sure you can supply plausible answers to both of those questions, but for the time being they are rhetorical & don't really need an answer. There will be benefits and penalties from any course of action.

If this announcement did not come so close on the heels of the Natural England MoA, maybe it would not make many of us in the community so nervous. Let us hope the consultation for this will be a more open and receptive one than some we have experienced of late. The fact remains that many of us work in organisations which have come about as a result of grass-roots activism and we are having to adjust to a world governed by top-down dictat. I don't think this is a question which will be resolved even in a five year timeframe. But I remain  interested to see where it goes.

As for the user group, yeah I am up for that. As for the rest, well we are going to be discussing this for some time yet, so I will keep a few arrows in my quiver for now.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

9

Re: Recorder 6 forward plan: position statement

I don't feel that I can say too much as I could be seen to have a vested interest in Recorder, but I would like to say that from the Consortiums point of view we are more than happy to listen to users views on priorities and in fact think we have done so. Sally carried out extensive consultation last year, and the work done on 6.19 was largely based on the priorities identified. To make best use of the resources, it is better to concentrate on one area at a time and 6.19 made as many of the import  wizard changes as we could afford. Germany also funds changes so their priorities need to be considered.  Keeping R6 running on the latest operating systems and versions of SQL are  also a priority and hopefully by the end of the year the few problems we have with SQL Server 2012 and  Windows 8 will be resolved.  This should put R6 in a position where is can be run for a number of years.   However, no system lasts for ever and as Charles says there are new approaches which need to be considered. What is unfortunate is that some of the wording in the annoucement suggests that R6 may not be funded even next year. Hopefully, this will not be the case.

Mike Weideli
Littlefield Consultancy - IT Consultants

10

Re: Recorder 6 forward plan: position statement

I don't think any of us would wish to denigrate the service You Jon and Sally provide in any way Mike. Without you, Recorder 6 would be dead in the water by now and I'm sure we are all grateful.

R6 is not without its flaws, but I have learned how to get what I want from it, and if I can't, I know you are there to help. Long may it continue.

I just find it hard to believe that a centralised, web-based system will be able to match the flexibility and versatility that is essential with such a diverse user group.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

11

Re: Recorder 6 forward plan: position statement

Indeed, many thanks to you and the others Mike - and especially to you and Sally listen to me witter on about what I think users want on more than occasion!

Here we have been really happy with the latest improvements. Being able to get taxonomy hierarchy info in reports has been so useful and time saving already this year (and makes it "almost as good as Recorder 3 again" according to a longer-standing user than me) and importing is much improved as well (and will improve again in the next version I think?). I think the latest improvements will really help us with updating the NBN Gateway, as we need to do over the next couple of weeks, much more straightforward than it has been in the past. :) If we can soon get species/person/location key recognition for to help us get data from online data collection systems like Rodis and indicia into Recorder more efficiently I would be a very happy record centre manager.

I never take the task of keeping the software working on the latest OS and DBMS lightly  - we have had a non-functional local sites database in the county for 18 months because of this type of issue that will take time and financial resources to sort out. This is the most important priority for users, that we can rely on continuing access to our data.

I agree the wording "this financial year" is rather alarming given that is only six months til next financial year, although medium-long term would imply otherwise?

-----------------
Teresa Frost | Wetland Bird Survey National Organiser | BTO
Other hat  | National Forum for Biological Recording Council
(Old hats  | NBN Board, ALERC Board, CBDC, KMBRC)

12

Re: Recorder 6 forward plan: position statement

RobLarge wrote:

I am less convinced however by the movement towards the cloud

I agree - it's a tachnobabblish, smoke and mirrors marketing buzzword of the most egregious kind. I firmly agree with this eloquent post on the subject and think it's a term to be avoided. Note that I very purposely put scare-quotes around the term in my post. Also, this. :) (apologies, I've just updated that last link - I pasted the wrong one before)

In fairness, the JNCC announcement doesn't mention the cloud, only 'online data management', which could mean so many things. It does't mention a centralised system either. The devil will of course be in the details, but I infer from the statement something based on open web standards, web services, and open source software. Does this mean Indicia? Perhaps, and if so, there is certainly no requirement for centralisation, or even access on the open web. It doesn't even necessarily mean the abandonment of a desktop tool with local database. It's just a more contemporary set of components and tools that are cheaper and more versatile than what we have currently. But as there's no mention of Indicia either, we'll just have to wait and see.

My sincere hope is that the considerable and broad knowledge of Sally, Mike and John are fully brought to bear in the consultation.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

13

Re: Recorder 6 forward plan: position statement

Rather than clutter this important topic I have started a new topic  covering the R6 points raised by Teresa. It would be helpful to know if what is proposed will meet the Rodis requirements.


http://forums.nbn.org.uk/viewtopic.php?pid=18363#p18363

Mike Weideli
Littlefield Consultancy - IT Consultants

14 (edited by DavidChun 01-10-2013 07:53:44)

Re: Recorder 6 forward plan: position statement

It takes avery long time from  the inception of a new system, until it is stable and accepted. I am aware of users who have devoted 10 years plus to getting their data into R6 and developing systems around it, based on the asumption that it was going to continue to be supported.    Agreed the JNCC statement doesn't actually say what is going to happen, but why make it unless the intention is to make cuts of some sort.  It doesn't make a lot of sense to me to stop supporting and developing Recorder until there is at least a detailed specification for the proposed replacement and there is general acceptance that this is a good way forward. 

Recorder 6 seems to have just about reached a stable position, but that doesn't mean it shouldn 't continue to be improved. Systems need to evolve to meet changing demands and I see from Mike's post above that 6.20 will introduce features which do just that.

This position statement would have been much better received and made more sense if it had invited comment on  proposals  for future development, without the implied threat to Recorder6.     
     
Dave

15 (edited by briantaylor 02-10-2013 09:23:43)

Re: Recorder 6 forward plan: position statement

I bought Recorder just over 12 months ago and was given the impression that it was well supported and continuing to be developed. I was looking for some help so have come into the forum for the first time only to come across this topic. Winter is approaching and I was planning to start adding more records from my spread sheets etc. Now I am unsure what to do.  MapMate which I looked at didn't seem flexible enough for my needs. Should I abandon Recorder and look elsewhere and if so has anyone any suggestion ?   I am going to take a lot of convincing before I even consider putting my data on an Internet system

Brian

16

Re: Recorder 6 forward plan: position statement

Thank you everyone for your comments and concerns. Please be assured that these will all be taken into consideration in forthcoming discussions on Recorder 6 development. There is no intention from any of the Steering Group partners to remove support for Recorder 6 without a viable alternative being in place. I am planning to meet with a number of you, or via a representative user group, to discuss your specific uses and requirements of the system. I will be in touch in due course.

Mary Campling

17

Re: Recorder 6 forward plan: position statement

Brian

I do not think there is a better alternative available at the moment and as several have noted, the likelihood is that nothing will change for the next few years. When (if?) it does, there will be a lot of users here ready to help you on to the next stage, whatever it may be.

Rob Large
Wildlife Sites Officer
Wiltshire & Swindon Biological Records Centre

18

Re: Recorder 6 forward plan: position statement

I run an LRC installation of R6 and also a private copy of my own (to handle Coleoptera records for VCs 61-65). Like others I have spent a very long time building up my databases from R3 in the early days to R6 now. I realise that we all have to move with the times and adapt to modern ideas (remember the dinosaurs?), but I feel that a purely online system of data input and handling to replace R6 would take away the complete control I currently have over my data, especially regarding validation and verification of records.  We also have to consider that the source of much of my LRC data originates in dedicated specialist amateur natural history recorders who choose to share their data with us and trust my handling of the results of their hard work. I feel that some at least might not appreciate their data finding a direct path to somewhere in the cosmos, and our sources of data contribution may be threatened. I have no objection to my/our records finding their way onto NBN having been confidently validated and verified. I find it difficult at this time to envisage a program which is as comprehensive and flexible (and even user friendly) as Recorder 6 in an online situation, especially in a 1 year or even a 5 year timeframe. Also I have little doubt that some recorders and LRCs may even move their databases to bespoke systems or even to other systems such as MapMate. I believe JNCC should continue to finance Recorder and continue to finance its development. R6 is now pretty stable  - OK, so the mapping is a bit slow – and establishing itself as a reliable standard, largely due to the dedication of the Sally/Mike/John team. Ultimately it is our money that is being used.

Bob Marsh
Doncaster LRC
Coleoptera recorder for YNU

19

Re: Recorder 6 forward plan: position statement

I don't mind whether I use a web browser or a thick client program such as Recorder 6. I do worry about things such as functionality, Ownership of records, distribution of records and availability.

I would not want loose any functionality that is currently available in Recorder 6.

An Internet only solution would be a degradation from the current status, as it dependent on Internet availability and the hosting provider ensuring the records are available when I want to access them. Fine if you don't suffer faults on the line, cable thieves or a poor service provider, and are actually working in a location that has access to the Internet.

There are legal considerations. When other people send me their records I keep their contact details in case I need to go back to them to check details. These contact details are covered by the Data Protection Act, which does impose certain constraints.

I distribute records to others, but I need to control when this happens (such as after I have validated the records), and in a format that they are able to receive. It would be great is everyone used the same recording software, but they don't.

If the records are hosted elsewhere other than under my control, who is responsible for backing up the data, and restoring the data in case of loss or corruption? Who pays for that service?

It would be good if there was a common interface that would work whether records were entered locally, or via a web site. However, it is useful to remember that there are different groups of people.
1) People who want to submit ad hoc records need an easy method for data entry which captures all of the relevant details.
2) People who want to maintain their own records as well as submitting records
3) The Record co-ordinator, such as the County Recorder who uses their local knowledge to check the records for errors and misidentifications. County Recorders are often unpaid amateurs with limited resources.
4) National groups or County Biodiversity will receive validated records from County Recorders. These will typically dealing with large data sets and be running complex queries against the data. This group are often paid professionals.
5) Researchers who want access to records such as from the NBN Gateway.

Harry Clarke
Surrey County Butterfly Recorder

20

Re: Recorder 6 forward plan: position statement

I understand that possible replacements for R6 are now being seriously condsidered again. As a user and someone who is directly involved with support of two other large systems I would make the following points which are  basically in line with those made by Harry Clarke,   

1. A great deal of time and effort has gone into getting records into R6 and it would be reasonable to expect that it should be possible to move R6 data to a new system without loosing any data. Many records have come been from earlier systems such as R3 without any loss of information,  so some R6 systems hold many years of data capture.  None of this information should be lost or become less accessible or maintainable in a new system.
2.  A replacement R6 system should not be for just the larger users (LRC's) , but  take into account the needs of the smaller users who  wish to manage their own data or have requirements not covered by one of the on line data capture systems. Individual users have tended to use more of the R6 features than LRC's for example that have used measurements for more than just abundance and have made use of the less well known aspects of R6 such as location features.
3. In any online system the data should be under the total control of the  user exactly the same as if it was on their own PC in much the same way as online accounting systems work. There should be the ability to download and remove the data in its entirety. Past experience has shown that it is not a good idea to be totally reliant on third parties because  however good the original intentions, political and funding issues can quickly change things.
4. As an R6 user who has shared my data with others through the R6 export process i expect  data custodianship rights to  respected and if data is moved  to a replacement system that it does not get manipulated or changed in meaning in the process and that the custodian can still be identified and update prevented.
5.Currently Internet  access even where I live in a town with cable can be very slow at times. I use an online accounting system which is much simpler than an R6 replacement would be and there are many occasions when this can not be used.  I believe the situation to be much worse elsewhere.  I am sure that over the coming years things will improve and that an online system is the way forward.  In the meantime R6 should be maintained and supported.

With a different hat on,  I can assure R6 users that just because possible replacements for R6 are being looked at this does not imply that the system is in any difficulty. In fact the opposite is true. It is in a  stable state running under Windows 10 and Sql server 2016. In the last two years over 20 new features have been added and many long standing bugs fixed. It would be reasonable  to assume that it  could be maintained  for at least the next 5 years and that it will still be usable  for many years after that.

Mike Weideli
Littlefield Consultancy - IT Consultants

21 (edited by DavidChun 11-05-2017 14:00:48)

Re: Recorder 6 forward plan: position statement

Thanks for the last paragraph. Why do you say that R6 can only be maintained for the next 5 years and what do you mean by maintained ? Don't you  think  that  by using R6 you put yourself in the hands of a third party ?

David

22

Re: Recorder 6 forward plan: position statement

R6 is just a  front end. You have full access to all the database tables, stored procedures and user defined functions and can access them using MS Access and other software with an ODBC link.  In fact Access can be used to write reports and it isn't that difficult to design an R6 input  form in Access. With an on line system much of the structure could be hidden so I personally would want the security of being able to get a flat file out of the system.

The 5 years is just being cautious. There comes a time in the time of a system where it needs to be replaced to take account of new technologies and requirements I don't think R6 is ant where near at that stage. It is nearly 4 years from the  original JNCC announcement and at that time we didn't know if R6 would run on Windows 10  and there were problems even with SQL Server 2012. R6 is probably in better shape than it was then, but who knows what Microsoft will throw our way. There may come a time when R6 will not run with the latest version of SQL server or it may require special installation to  run with the latest MS Operating System, and it this point it may not be worth modifying to deal with this.  However, I would expect it to remain useable long after that.

Mike Weideli
Littlefield Consultancy - IT Consultants

23

Re: Recorder 6 forward plan: position statement

MikeWeideli wrote:

However, I would expect it to remain useable long after that.

And Recorder 6 works perfectly well inside a virtual machine (just like Recorder 3 - how old is it now?)

Charlie Barnes
Information Officer
Greater Lincolnshire Nature Partnership

24

Re: Recorder 6 forward plan: position statement

Recorder 3 was developed in the late 1980's.

Mike Weideli
Littlefield Consultancy - IT Consultants

25

Re: Recorder 6 forward plan: position statement

Thank you Mike; I fully support and endorse the points raised in your post.
As a database manager for a small voluntary biological recording I would not have serious reservations about a purely on-line system. Systems designed for large users with professional paid staff and superfast broadband very rarely consider the needs of the small user with limited resources.
Naturally we have to be prepared to accommodate changing technologies, but I hipe that those tasked with introducing changes or designing a new system, listen to the needs and requirements of all users and not just the larger and more influential users. It is equally important that someone one the team has a knowledge of biological recording and taxonomy in the UK to prevent the problems that have arisen with the Atlas project.

Christine