Topic: NBN Atlas performance and stability problems update
As you will be aware from previous updates (April update and June update), we have been experiencing problems with the NBN Atlas for a while. Over recent months the records and species searches have been running slowly and often failing to return results with a timeout error. The two searches run off separate Solr indexes and in February we engaged a Solr Specialist company to perform an audit on our current Solr implementation and make recommendations on how to improve the performance and stability of the NBN Atlas searches.
The recommendations made relate to, but are not limited to, the design of the index (schema), the size and number of the servers (cluster) used to serve the index and increasing the memory size.
To date, we have been unable to utilise the schema changes and changing the size and number of the servers has not made a significant improvement. We are still working with the specialist company to see if increasing the memory available to the search functions will resolve the instability. This is a multi-stage process with frequent testing of each solution to ensure compatibility, which means we are not yet able to give a firm completion date, but we are progressing as swiftly as possible.
The purpose of this latest update is to give you a timeline of what we are doing to rectify this, both for the short and long term.
Short-term, we are working with the specialist Solr company as reported above.
For the longer-term, we have commenced a detailed investigation and options appraisal to minimise the risk of future relapses. We need to make sure that both the system architecture and the codebase are fit for the future. We expect this project to run through to November, and we will keep you updated with its progress – through Network News and a dedicated project page on the NBN Atlas.
We know that the unreliability of the Atlas has been extremely frustrating for our community of Atlas users. Our small team has been working really hard, to address the problem, and we are very grateful for the patience and understanding of those affected. We would like to apologise if your work, or your colleagues’ work, has been hindered in any way.
We would like to reassure you that the performance of the Atlas – both now and into the future – is of utmost importance to us. We’re doing everything we can to make sure that the important service the Atlas provides is dependable, resilient and of the highest quality.
As always, we thank you for your patience and support.
If you have any comments or questions, please do get in touch with us firstname.lastname@example.org