1 Topic by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Topic: Multiple record upload

Someone would like to upload multiple (60+) records to the PlantTracker DB.  Can I do this directly from the Indicia warehouse by uploading a csv file?

If so, I'm assuming the data must be in exactly the right format with correctly specified row headers and number of columns.  But, how do I find out the structure required?  Also, how do fields like ID# become populated?  Would that have to be a manual operation?

Cheers,
Dave

2 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: Multiple record upload

Hi Dave,

Yes, you can upload from a CSV (I've just uploaded nearly 400K records into our warehouse via this route). The data doesn't have to be in a precise format, you just need columns in the CSV that will map to columns in your database. The importer has a UI step that enables you to do this mapping. This is what it looks like:

http://i.imgur.com/8QuZ7.png

Here are some docs, although they're a bit light on details:

http://code.google.com/p/indicia/wiki/WarehouseImport

Feel free to ask any further questions.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

3 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Multiple record upload

Hi Dave
There is a new documentation site for Indicia, still work in progress but it will hopefully flesh out quite a lot of things going forward. Here's the link to the relevant section on importing: https://indicia-docs.readthedocs.org/en … rting.html

Note that there is a prebuilt form in Drupal which you can configure for importing as well so you don't need to go to the warehouse.

Best wishes

John van Breda
Biodiverse IT

johnvanbreda's Website

4 Reply by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Re: Multiple record upload

Thanks both - I'll have a go and see how I get on.
Cheers,
Dave

5 Reply by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Re: Multiple record upload

Hi,

The upload wizard is quite straight forward to use but I hit this problem:

Could not find a Taxa Taxon List by looking for "japanese Knotweed" in the Taxon field.

Any ideas?  I have attached some screen grabs and the .csv in the zip

Cheers

D

Post's attachments

test-upload.zip 140.83 kb, 5 downloads since 2012-11-29 

You don't have the permssions to download the attachments of this post.

6 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: Multiple record upload

Hi Dave,

The importer is trying to look up the taxon names in your csv file to find a match in your species list. It has failed. I'll bet you a slice of pizza that if you make the 'J' in 'Japanese' upper case then it will work.

Jim Bacon

7 Reply by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Re: Multiple record upload

Hmmm, tried that already :(     I'll give it another shot though just incase I mistyped something else...

8 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: Multiple record upload

Hi Dave

Show me an error message with a capital J in it and you win your slice!

Check for any spurious space characters at start and end of the taxon name in both csv file and taxon list, plus mismatching space between words too.

Jim Bacon

9 Reply by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Re: Multiple record upload

Hi Jim,

I've retried it again and now it's saying "Please specify a value for the Amount."  No complaint about the taxon name this time so I think you were right and I obviously made a mistake twice with that before.  Any idea why it can't match the amount attributes? The case is correct in the .csv looking at the db.  It's one of our Occurrence Custom Attributes and it's listed in the bulk uploader as "Amount (lookup existing record)".  Could that affect it?

Cheers,
D

10 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: Multiple record upload

Hi Dave,

I would expect you would get an error like that if you forgot to assign the "amount" column in your csv file to the "Amount (lookup existing record)" or not every record in your file had an amount value.

The 'lookup existing record' means look in the "Amount" term list for the value in the csv file which is what you want. It is a required attribute which is why it must be mapped to a column in the csv file and every record must have a matching value.

Last help from me till Monday. Don't let it ruin your weekend ;-)

Jim Bacon.

11 Reply by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Re: Multiple record upload

Hi Jim

I've tried everything I can think of and still no joy.  I've attached the error report which clearly shows that the amount column contains a value and these values are (to the best of my knowledge) correctly specified.  No missing values, no typos etc.

Any ideas?

Cheers,
D

12 Reply by Jim Bacon

  • Jim Bacon
  • Veteran
  • Offline
  • From: BRC
  • Registered: 03-11-2008
  • Posts: 980

Re: Multiple record upload

Hi Dave,

No attachment visible. May be there is a bug. Regrettably my quota of time for looking at this forum has already been exceeded by answering your other questions! Unless someone else can take this on it will have to wait a little while.

Jim.

13 Reply by dave kilbey

  • dave kilbey
  • Member
  • Offline
  • Registered: 14-08-2012
  • Posts: 29

Re: Multiple record upload

Error was (of course) a user error.  The taxon field of the .csv contained an entry which didn't match the species list in the warehouse.  Once this was corrected the process worked perfectly.  Thanks to Jim for spotting this.