1

Topic: Error when entering large numbers of records

One of my users has noticed that when she tries to enter a lot of records (using the 'Enter a list of records' form on SEWBReCORD), the website throws an error stating that the data could not be saved, a spatial reference must be supplied. This usually happens after a successful list of data has been entered, but can occur first time. This is clearly frustrating for the user, who is trying to get into the habit of submitting lesss records in one go.

I have not yet attempted to replicate the behaviour, but thought I would ask to see if anyone else had experienced this problem or had any suggestions to the cause?

Dave

Dave Slade
Senior IT & Records Officer,
15 Talbot Road, Talbot Green, Pontyclun, CF72 8AD
www.sewbrec.org.uk, www.sewbrecord.org.uk

2

Re: Error when entering large numbers of records

Hi Dave,

Presumably the user has definitely supplied a spatial reference and this appears to be a fault rather than operator error?
Am I also correct to presume that the list of species is somehow lost so it is more frustrating than just having to add a location and resend?
How many is 'a lot' of records? Are many photos included?
Is it repeatable if the exact same list is entered again?

I recall encountering a problem once when long lists of species were being entered in one go. That was due to PHP limits on the Drupal server (e.g. post_max_size). However, I think that would result in a more brutal error than the one you are seeing. Still, worth checking the php error log around the time of a reported problem.

Jim Bacon.

3

Re: Error when entering large numbers of records

Hi Jim,
I don't think that photos were generally being submitted - this user seldom supplies them.
The error in question appears to break the session as far as the user is concerned and required them to restart their web browser, thus losing the species list that had been entered. The spatial reference had been supplied - usually be selecting the appropriate spot on the map interface, so it shouldn't be user error (at least not that part of it).
The number of records was thought to be between 50 and 100, but she didn't actually count them.

I will try to investigate further and get back to you if I work anything out.
Dave

Dave Slade
Senior IT & Records Officer,
15 Talbot Road, Talbot Green, Pontyclun, CF72 8AD
www.sewbrec.org.uk, www.sewbrecord.org.uk

4

Re: Error when entering large numbers of records

The error in question appears to break the session as far as the user is concerned and required them to restart their web browser.

Having to restart the browser implies that the browser has crashed. That we might achieve that seems highly unlikely. I presume it is sufficient merely to navigate back to the website and make a clean start.
I also presume that, when starting again, it is not necessary to log in to Drupal again. If you need to log back in to Drupal it would imply cookies have gone missing and the Drupal session has broken. Again, very unlikely.

The problem is almost certainly with our code. When records are submitted the browser sends information to the Drupal site and the Drupal site forwards it to the Warehouse. The Warehouse replies to the Drupal site and the Drupal site replies to the browser.

One way this can break is if the user spends too long entering data. Each recording form is authenticated with the warehouse to send data for a limited time. However, this results in a not-authenticated error, not the one you are seeing.

Client Side Validation should be enabled for your recording form (a check box under User Interface options). This should ensure required fields (spatial reference is required) are filled in before the form can be submitted.

If a valid form is submitted then it should be forwarded correctly and accepted by the warehouse. If the warehouse detects an error in the submission, as seems to be the case here, then it sends an error response to the Drupal site and the Drupal site is supposed to send the erroneous form, with all the original data back to the browser so it can be corrected and resubmitted.

This suggests that the submission is being mangled somewhere on its journey which possibly makes it unsurprising that the response is also broken. My instinct is to trust the warehouse and to look for the problem in the vicinity of the Drupal site. You know that the form typically works and you associate the problem with a long list.

Like you say, I think you will have to investigate further. You could create a record with a dozen occurrences and send it. If successful, edit, add another dozen and send again. Repeat a number of times and see if it breaks. Probably it will. Probably you will then find you have a repeatable error. Probably we'll then be able to see what it is. Whether it is the same error that your user is experiencing is another question.

Let us know how you get along.
Jim Bacon.