1 Reply by Andrea Sforzi

  • Andrea Sforzi
  • Member
  • Offline
  • From: Grosseto, Tuscany, Italy
  • Registered: 16-04-2012
  • Posts: 32

Re: problems with the records upload procedure

Dear Indicia users,

We created a page accessible from the home page to access to a specific survey, with all the distributional data of a target species, at the national level. We wish people to use this page as a "living map" dedicated to this species and eventually contribute with their own observations.

We created (1) a specific survey; (2) a species list with only our target species; (3) a .csv file with locations of sigthings, like the following:

Taxon, X, Y, location, type, date
Istrice, 39.793372N, 9.517186E, PELAEDDU, Introdotto, 02/05/2012

in order to upload the current data.

During the upload, we matched the fields as follows:
Taxon = Species or taxon selected by extisting list;
X = Grid ref or other spatial ref ;
Y = Grid ref or other spatial ref ;
Location = Location name ;
Type = Comment ;
Name = Name .

Unfortunately, we didn't succeed to upload our file and an error file was created by the system. For each record, the message was the following:

Taxon,X,Y,luogo,tipo,date,Problem,"Row no."
Istrice,39.793372N,9.517186E,PELAEDDU,Introdotto,02/05/2012,"Please specify a value for the Tipo di dato
Please specify a value for the Sesso
The spatial reference is not recognised.
The spatial reference must be supplied.",2

Most likely, we will have not to set as "required" the fields "Tipo di dato" and "Sesso" in the settings of our dbase.

However, what about the error with spatial references? Did we prepare correctly the .csv file ? Did we write spatial references in the correct way? Did we match them with the correct field?

Thank you very much to whom will answer!

Andrea Sforzi and the Naturae Social Mappping team

Andrea Sforzi's Website

2 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: problems with the records upload procedure

Dear Andrea,

The spatial reference is a single field in the database so it must be a single field in your csv file.
Simply change your csv to read as follows and it will work.

Taxon, sref, location, type, date
Istrice, 39.793372N 9.517186E, PELAEDDU, Introdotto, 02/05/2012

Jim Bacon.

3 Reply by Andrea Sforzi

  • Andrea Sforzi
  • Member
  • Offline
  • From: Grosseto, Tuscany, Italy
  • Registered: 16-04-2012
  • Posts: 32

Re: problems with the records upload procedure

Dear Jim,
I modified the record as you suggested (and added the two fields that gave error messages), but the
"The spatial reference is not recognised" error still persist. Below you'll find the current structure of data:

Taxon,    XY,    luogo,    tipo, data, Tipo di dato, sesso
Istrice, 39.793372N 9.517186E, PELAEDDU,    Introdotto, 02/05/2012, Avvistato, Indeterminato

I tried to change a little bit the lat-long string adding and cancelling spaces, but apparently there is no way...

Any further suggestion?

Andrea Sforzi's Website

4 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: problems with the records upload procedure

Hi Andrea
If I change your taxon to one from my own list, then I can import your data file OK. When you do the import are you setting the Spatial Ref System on the first page to "Latitude and Longitude (WGS84)" and matching the XY field to "Sample > Grid ref or other spatial ref"?

John van Breda
Biodiverse IT

johnvanbreda's Website

5 Reply by Andrea Sforzi

  • Andrea Sforzi
  • Member
  • Offline
  • From: Grosseto, Tuscany, Italy
  • Registered: 16-04-2012
  • Posts: 32

Re: problems with the records upload procedure

Yes, John, that was the procedure I followed.
Since I have the perception that some problems would be in some way related to the difficulties encountered in managing the Italian drupal menu, I asked our technicians to switch back to the English version. I'm not sure that it will solve this specific aspect, but 1) will prevent further problems; 2) will permit a simpler communication among us, by citing the same terms referring to the description of a particular aspect.

Andrea Sforzi's Website

6 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: problems with the records upload procedure

Hi Andrea
I doubt the Italian Drupal translation would affect this but it does help communication I agree. Do you mind trying a bit of debugging please? Can you find the file on the warehouse application/helpers/spatial_ref.php then open it in a text editor. Now search for the declaration of the function process_lat_long. At the top of this function there is a comment, then 2 lines of code which set 2 variables, $latitude and $longitude. Please can you insert the following on the subsequent lines before the line that sets the $results variable:

    kohana::log('error', "Lat: $latitude");
    kohana::log('error', "Long: $longitude");

Now run your import, then go to application/logs/ and find the log file with today's date. Open in a text editor and search for the lines that contain Lat: or Long: and copy and paste them into your reply. That might give me a clue where the problem is so we can debug further.

John van Breda
Biodiverse IT

johnvanbreda's Website

7 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: problems with the records upload procedure

Hi Andrea
Looking at the logs, I realise the error is probably simply the Italian regional settings on the server. The lat long format must be written in Italian style, i.e. rather than write
39.793372N 9.517186E
you should write
39,793372N 9,517186E

Notice the . has been replaced by a comma.

Let me know if that helps,
Best wishes
John

John van Breda
Biodiverse IT

johnvanbreda's Website

8 Reply by Andrea Sforzi

  • Andrea Sforzi
  • Member
  • Offline
  • From: Grosseto, Tuscany, Italy
  • Registered: 16-04-2012
  • Posts: 32

Re: problems with the records upload procedure

Thank you John,
to avoid current, past and future problems related to the translation in another language (or, better management of two languages at once) I asked to our technicians to set English as the only language used by Drupal.
Once did that, I supposed that the upload problems would have been solved. But I wasn't right. The error file ignore the coordinate problem (probably solved at this stage) but reports the input of an invalid value for one of the ancillary informations (incuded as a term in the term list). I checked for the spelling and for the capital letter at the beginning of the word, but everything was apparently o.k. I'm getting crazy!

Andrea Sforzi's Website

9 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: problems with the records upload procedure

Hi Andrea
I don't think the spatial reference import issue related to the Drupal language settings, though maybe the language settings on the actual Indicia warehouse server.
So, are you saying that:
1)  you are importing a csv file with a column that contains a value which matches a term in a particular termlist,
2) when you import the file, you match this to either a sample custom attribute or an occurrence custom attribute which is a lookup, using the same termlist as mentioned above?

If so, please can you send me the log file for the import (from application/logs on the warehouse).
Best wishes

John van Breda
Biodiverse IT

johnvanbreda's Website