It's not clear from this message what the definition of "online recording" is. It would be useful to clarify what is meant so that we can all be on the same page in ongoing discussion. Does it mean:

* A centrally hosted and managed system
* A self-hosted system (that could be run on an intranet as opposed to "online")
* Something else

Charles

Charles,
My apologies for the ambiguity. There are a number of tools and toolkits available for capturing and managing biological data in an online environment. These vary from centrally hosted and managed systems to self-hosted intranet or linked systems. We are considering all possible options, both self-hosted and centrally hosted, and hope to adopt or develop a solution that fits as many use cases as possible. I will certainly provide more information as soon as it is available.
Mary

Thank you for the update Mary.

You may be interested to know that ALERC has already started consulting our members on this topic. It is important to us because of the 37 members who submitted statistics, 35 are using R6, with only two having their own bespoke databases and none using alternatives exclusively, although some use additional recording software such as Mapmate for records exchange. We have had a good response to the consultation and will use this in ALERC comments to you. If any ALERC member reading this didn't contribute and now wishes to, please contact Tom.

I consider it essential for any consultation and cost-benefit analysis to consider the costs involved for all third and public sector users, which as Rob says are often integrated with other software. One of the benefits of Recorder 6 is that it is used by organisations such as Wildlife Trusts, museums and other conservation and land management charities and the professional data collected is just as valuable to our collective knowledge of the UK's biodiversity as that by volunteer recorders and citizen scientists, and because it uses Recorder it is collected to NBN standards and the NBN data model. It is not just about species determinations, it is about managing locations and sources too. For many of these groups, such as LRCs, increased costs if they need to find an alternative system as customisable as Recorder integrated into the way they manage other environmental datasets as Rob explains, is likely to lead to less efficiency in the way public money is spent (including from the Defra agencies, local authorities and project funders such as HLF).

Another group it is necessary to consider is individual recorders using R6. This is a rather select group when considered in the number of people who submit any biological records in totoal, but certainly in my experience the couple of people using R6 personally in the counties I have worked in are also the most profilic recorders in the under-recorded groups that are the focus of much public money at the moment (invertebrates, fungi, lichens etc.). They fill in the gaps, and they are not the ones who are using the new online recording tools because their data is complex, extensive, and important to them so they want to keep control of it. In April's National Federation of Biological Recorders conference, when the question was asked about who was using Recorder, the majority of the room were.

Agreeing with Janet. Butterfly Conservation is also committed to Recorder 6.

Also consider (like us) that not everyone has 21st Century broadband speeds. Here in Lulworth the standard download speed is 300Kb/s, therefore we have to at great expense strap a number of line just to make the Internet usable for our business. I would very quickly annoy all my colleagues if I were to be seen to hog the Internet access importing 100,000's of records daily to a remote system.

Les

On-line recording might be fine if you live in an area that has fast and reliable broadband, for those in rural areas it will be not be a feasible option.
Many biological recording groups are run by volunteers, often without funding, and Recorder 6 provides a reliable and flexible platform for building and maintaining databases. It provides the mechanisms we require for data handling, verification and analysis. No fuss, no bother and no down time - unless we have a power cut, which are not infrequent in this part of the world.
It would be a retrograde step to move to the one-size fits all solution, it won't and could be to the detriment of the smaller groups whose contribution is often greater than their size.
More technology is not always the answer, one only has to look at the NBN Gateway 5 fiasco!

Christine Johnson
Outer Hebrides Biological Recording

Looking at the Microsoft Lifecycle page http://support.microsoft.com/lifecycle/ … ;alpha=sql it looks like support for SQL Server 2000 has finished, 2005 will be supported up to 2016, 2008 to 2019 and 2012 to 2022

To me this seems like a measured, appropriate plan. JNCC nor any of us can ignore the seismic shifts in IT toward greater connectivity and 'cloud'-based services. We cannot simply hope it will go away. We all need to look forward and plan for the future. It seems wise to me that JNCC are carving out some budget to plan ahead, investigate options, engage in dialogue. We must not leave it any longer.

Don't forget, it was only in 1999 that broadband was introduced and since then we've seen over a 100x speed increase (512 Kbps vs the now top-end 77,824 Kbps on fibre to the cabinet). My first non-broadband internet connection in 1994 clocked at 56 Kbps. We've seen the introduction of 3G mobile broadband, with 4G being gradually rolled out. And new technologies are in constant development (e.g., http://www.theguardian.com/technology/2 … less-rural). When I started at SxBRC in 2003 we had a miserable 64 Kbps (!) connection thanks to our rural location. We now have a rock-steady ~10,000 Kbps connection. So although we shouldn't be complacent, we have reason to be optimistic over the timescales we're talking about (5-10 years).

I also think it's worth defining clearly what is meant by 'online' because it is a vague term open to misinterpretation. In the sense JNCC are using it, I believe they are referring to a client-server architecture where the server is a RDBMS (e.g., SQL Server, MySQL, PostgreSQL) controlled by a server-side language such as PHP, and the client is based on web standards (i.e., HTML, CSS and Javascript) and runs in a web browser. It may be that the RDBMS is hosted remotely ('centrally'), but it does not have to be. You could host the client and the server on the same machine. An installer could be provided to set it all up without requiring specialist expertise - as is the case with Recorder now. Not a problem. Or the server could be within your local LAN. Recorder already does this and offers either configuration. Difference being with Recorder is that the client is coded in Delphi as a 'native' desktop application rather than HTML/CSS/Javascript/PHP. The latter provides much greater scope for internet connectivity and open source collaboration, and is MUCH more future friendly.

That said, many of the advantages Mary cites in her message may only be realised if the server component of the system were hosted centrally. If an individual were to host the server locally, the maintenance burden could be just as great as with Recorder, if not greater, and the seamless collaborative potential would be limited. However, a well designed system could be much easier to maintain, with updates able to be pulled automatically from the internet and installed with minimal user intervention. Sync mechanisms could be developed to push and pull data in a distributed rather than central fashion. Technologies such as Riak[1], Cassandra[2], Git[3], Github[4] and GeoGit[5] (to name but five) could be investigated for managing this kind of workflow. These are the sorts of requirements that need to be explored.

[1] http://basho.com/riak/
[2] http://cassandra.apache.org/
[3] http://git-scm.com/
[4] https://github.com/
[5] http://geogit.org/

The major advantage of a web-standards based system is compatibility with a wide range of systems and devices. If the system is capable of running a standards-compliant browser, then it can use the application (in theory). The disadvantage currently is that 'native' applications tend to have a richness and speed that HTML-based applications cannot quite match. Yet. But this is changing remarkably quickly. For example, try https://www.lucidchart.com/ or https://office.com/. Offline capabilities in browsers are ever-improving, allowing work to be done locally and synchronised later. So a question to ponder: what in Recorder is possible now that you feel would be impossible or sub-standard in the browser?

Make no mistake, the full competitive energy of the IT world is focussed toward online technology. The line between what is local and what is remote will continue to blur and eventually be gone. There is a convergence happening as neatly exemplified in this article: http://boundlessgeo.com/2014/05/spanning-globe/ and in this talk https://www.youtube.com/watch?v=01yrhqCro7I (sound quality poor, sorry). The most important considerations for me are not so much about where data is stored and manipulated, but about data security, preservation, ownership, privacy and control. Connectivity, standards, interoperability and openness are paramount, as is machine-readable metadata. And also, of course, a smooth, manageable transition. Funding development of a new system is not enough - resources also need to be made available to help those with bespoke systems based on the Recorder ecosystem adapt. Recorder will - and should - be around for a long time yet, but it's vital that we plan now for the future and work hard to embrace it. We're already leaving it a little late as it is.

Charles' latest post reads like a chapter in a project document, perhaps that's something that could be usefully put together, incorporating all the feedback from users, both current and potential.
For the record, Dipterists Forum recording schemes have 4/18 R6 users, two ex-LRC users, one who wrote the first Recorder (Stuart Ball of the Hoverfly scheme) and one brave soul; the rest are "potentials" (although one of them is Charles' dad).
ALERC's national coordinator is currently asking what LRCs are using R6 for, which raises the question of what it cannot be used for (yet should).
My concern is that any development, in whatever form, is simply going to take the current model and run with that. Charles Copp would have been the first to acknowledge that his model was incomplete. After all he wrote an additional module, not incorporated into R6, to deal with specimens. Since then other inadequacies of the model have been highlighted:
1. Metadata - this was just a bolt-on, there's no comprehensive metadata module, just a cut-down geospatial one that only partially conforms to published standards, doesn't store on the originators PC and is lost each time datasets are updated.
2. References/publications - this works badly, if anyone tells Tom Hunt they use R6 to keep a record of these I'd be surprised, EndNote and Mendeley are leaders in the field and show how this kind of thing should work (the latter is online). A much improved system would attract more users.
3. GIS - there'd be no need for that "mapping unit" (Hannah Betts' words - Recorder tech from way back) so I'm guessing something else is going to be used in the online version here. But take a look through recent publications that are related to biogeography and assess their quality and methodology (try British Wildlife for people's best efforts). These vary from text descriptions (unacceptable) to pretty good renderings using NBNG or R6 output into DMAP. The best achievable is a GIS app output (free QGIS) into pdf, which is the same format as Adobe's vector graphic, meaning you can fiddle with shades, keys and scales to your heart's content. Wiith Adobe Illustrator + QGIS you've got the absolute top quality maps, and yet I've never seen a map done that way in a publication. So the GIS component should be in three parts, one as a location finder (with GPS unit reading facilities built in - like Garmin's Basecamp) and record cleaner, one as final online display (that's NBNG) and one for the home user to produce maps of publishable quality; recording schemes will always be ahead of NBNG and want to show things that NBNG cannot - which leads me on to ...
4. Capacity for recording outside the UK. Sure this brings up species dictionary problems (they are online though) but there simply isn't any means of recording elsewhere than in UK. Most recording schemes will have taken an interest in the European distribution of their taxa or gone abroad and recorded, just look at all the data on the photographs posted onto online ID sites (e.g. Diptera.info) all going to waste. GBIF is where we'd like it to finish up but posting anything there is a nightmare (Darwin Core format)
5. Specimens - see Charles Copp's model used in some museums in Europe. This bit would probably include photographs too - a growing source of records (see iMatch)

Dear Mary,

It is with great interest and concern that the LRC community has read JNCC’s announcements on Recorder 6 and its future.  Having digested these, ALERC has prepared a response which offers information and guidance and that we hope notifies you of the key issues that will need to be addressed in the not too distant future.

ALERC has information on the biodiversity databases of 39 of its member LRCs.  Of the 39, 34 (87%) use Recorder 6.  Therefore, it is safe to conclude that Recorder 6 is very much the main database software of choice for LRCs.  According to figures from 2013, the LRCs using Recorder 6 are storing nearly 64 million records in it.  These figures are due to be collected again this summer, at which point we expect to see even more records being held in Recorder 6 up and down the UK.  This is significant number and shows how important Recorder 6 is for maintaining and sharing on the UK’s biodiversity.

Given the large stake that LRCs hold in the future of Recorder 6, ALERC feels that decisions made on its future must be taken extremely carefully in order to ensure that data sharing continues to work for the benefit of data providers and data users.  Therefore, a lot needs to be considered before decisions can be made.  This is outlined below.

Current status of Recorder 6

The sheer volume of data that is held within Recorder 6 databases shows that, whilst it has room for improvement, Recorder 6 provides an essential service.  We know that a vast number of species observations are held within Recorder 6 databases, but this is only part of the story.  Holding species observations is just one function that Recorder 6 performs.  It is extensively used to hold site and habitat information as well, and has the capacity to store detailed information on recorders, which can be vital for verification purposes.

One of the most important considerations is that Recorder 6 can be operated in different ways depending on local circumstances.  This is particularly true for the way data is arranged within the database.  LRCs receive data from different sources, for example one LRC may have most of its data provided by a local natural history society in bulk uploads, whilst another may have to accumulate data from a number of different species interest groups or field survey teams.  Recorder 6’s system of storing data in individual surveys can be used to great effect in these situations, allowing sections of data to be imported, updated and exported in an efficient and organised manner.

The affordability and customisability of Recorder 6 means it is used by local voluntary recording groups, small conservation and community groups and individual recorders to manage their own data who then pass records on to their LRC. This helps efficient data management as data arrives at the LRC in a standard format using the NBN taxon dictionary, rather than spreadsheets and tables that take time to process.

Preserving functionality

ALERC welcomes the fact that JNCC only wish only to cease support for Recorder 6 once, or if, online applications are able to provide the same level of functionality and accessibility. As suggested earlier, Recorder 6’s extensive functionality can be used in different ways to suit different local situations, which LRCs have utilised to maximise the benefit for their recorders and clients.  For example an LRC hosted by a museum may be responsible for logging specimen numbers in a collection.  In fact, use of the Collection Module enables museums and collections use Recorder 6 as a complete database for all their collections.

Systems for reporting are another key consideration for LRCs.  The combination of the species data, habitat data and site boundaries that an LRC has access to, plus the format that these are provided in will all determine how an LRC is most efficiently able to produce reports for locations in its area.  These variables are also combined with the fact that there are varying GIS platforms that LRCs have available to them, as well as varying programing expertise and vary demands from clients, such as local authorities.  The result of this high degree of variability is to produce many different reporting routines and methodologies that are tailored to differing local situations.  Recorder 6 allows LRCs to do this in a number of ways.  Many LRCs have developed semi-automated routines, often using database management systems that allow records and other site information to be opened in a GIS, thus allowing for the added functionality of the GIS to be used to query the records in complex and sophisticated ways.  These routines, which manipulate records via Recorder 6’s “back end” bypassing the user interface, are very common amongst LRCs.  However, in many circumstances, Recorder 6’s own reporting functions are also used.  This would be done where it is quicker and more efficient to compile a report using Recorder 6 itself, although further formatting might be required after the report is originally produced in order to make sure it is presented in the desired fashion.

Another major consideration is the fact that LRC staff are competent in using Recorder 6 and familiar with its interface, its structure and its quirks.  The introduction of a new system means that time and resources will have to be found in order to get LRC staff up to the same level of competency.

Networking and online capacity

Recorder 6’s networking capabilities are of prime importance to LRCs, who operate in an environment that may mean their database needs to be accessed from different work stations for different reasons within an office environment.  Moving to online systems will add to this and increase flexibility.  In many cases, online recording systems are already used to increase efficiency and flexibility for data input, allowing recorders to submit their records directly into a database if they want, and managing them there.  The important thing to realise when planning the future of recording software is that these systems are used in conjunction with Recorder 6.  They don’t have the functionality that Recorder 6 has, but offer a different set of opportunities such as remote data input, automatic validation and remote verification.

A recent ALERC consultation found mixed views about the benefits of a transition to online-only solutions.  Where people are concerned about the prospect, they raise a number of issues, all of which will need to be addressed by JNCC if they are to produce a new solution for biodiversity recording.  These issues include technical concerns regarding the “up time” of websites and importantly the reliability of internet connections.  LRCs who see online solutions as more beneficial tend to be those already using an online recording system.  In these cases, the potential efficiencies for data input, validation, verification and reporting are recognised.  However, it is important to note that in these instances the LRCs are using a combination of online recording software with Recorder 6.  In some instances, they may even use two or more online recording websites, chosen to appeal to different sections of the recording community or for work on different projects, but all of which provide data stored in Recorder 6.  Details and case studies of specific LRC online recording implementations can be provided on request.

With all this in mind ALERC would like to recommend that JNCC publish and consult on its vision for the future of biodiversity recording as soon as possible.  This will allow LRCs and their partners to comment in a constructive way and provide the information necessary in order to produce a strategy that will meet the needs of JNCC and all the others who currently hold a stake in recording software.  Publication will also have the added advantage of clarifying exactly what is planned and disabusing people of any false notions that exist.

The future

Little is known about JNCC’s plans for the future of biodiversity recording software, other than the intention to withdraw support from Recorder 6 and any other versions of Recorder.  ALERC welcomes communication at this early stage and would welcome further communication and consultation, possibly including workshops as well as written work.  It is important that future solutions include the functionality outlined above, as well as allowing easy migration of both data and reporting routines into new systems with 24-7 accessibility.  The danger if this is not done satisfactorily is not so much that there will be ineffectual or broken systems in place, but rather that LRCs will not migrate at all and any resources used to develop new systems may be wasted.

With all these considerations, it is likely that a good deal of time would be required in order for LRCs to migrate to a new system, assuming they wish to.  This is not simply because of the resources it is likely to take, but also because the changing technological environment and the various hosting arrangements (for example local authority hosted LRCs often have difficulty obtaining permission to access certain websites).  Not everyone as yet has access to a satisfactory broadband connection, vital for migration to an online database.

Should you require clarification or further information on any of the points raised, please do not hesitate to get in contact.  This is a very important subject for most LRCs, and as we represent a major Recorder 6 user group, ALERC would welcome the opportunity to discuss it with JNCC and others.

Tom Hunt, on behalf of ALERC

With financial constraints already threatening the future of our LERC, things are now looking decidedly bleaker after the announcement of the pulling of support for R6 early in 2018. The cost implications of adopting and moving our data holdings to an alternative system may well be the final straw as a Local Authority hosted LERC!