1 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: Import wizard and multiple recorder delimiters

When importing data from Mapmate, I frequently see this sort of thing in the Recorder field:

Mr and Mrs J Bloggs
Dr E O and M Wilson
Bruce & Dorothy Wayne, John Doe
D & P Green

And so on. The problem with this is that the import wizard parses the "and" and the "&" as a delimiter and thus thinks we are trying to import:

Someone called "Mr"
Mrs J Bloggs (correct)
Someone called "Dr E O"
M Wilson (correct)
Someone called "Bruce"
Dorothy Wayne (correct)
John Doe (correct)
Someone called "D"
P Green (correct)

I currently have to identify these troublesome names and fix them, which, even with a large degree of automation, is a huge pain. The Mapmate documentation recommends that users use the above convention for recording names to save space, so it is unlikely this issue will go away any time soon.

I think the only way around this problem is to allow for multiple recorder name parsing to be turned off for a given import. I have a sneaking suspicion this might have been suggested anyway as a way of speeding up the import wizard, but I thought I'd best make this suggestion here anyway.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

2 Reply by johnvanbreda

  • johnvanbreda
  • Administrator
  • Offline
  • From: UK
  • Registered: 27-04-2006
  • Posts: 1,237

Re: Import wizard and multiple recorder delimiters

Another suggestion, do you think we could add a rule to the Import Wizard that says if a name does not appear to have a surname, but is followed by an "and" or "&" then a name that does have a surname, then it could use the following name's surname? The only problem with this is names of the form "Wayne & Dorothy Brown", where the system would not know if the Wayne was a surname of someone unrelated, or firstname of someone with surname Brown. However, I'd hazard a guess that this is much more likely to be the latter since names in an import are likely to take the same form - so perhaps the wizard could create the name but highlight the assumption somewhere for you to double check.

John van Breda
Biodiverse IT

johnvanbreda's Website

3 Reply by charlesroper

  • charlesroper
  • charlesroper
  • Digital Development Manager
  • Offline
  • From: Field Studies Council
  • Registered: 26-03-2006
  • Posts: 906

Re: Import wizard and multiple recorder delimiters

Excellent suggestions. Given the problem cases I've been coming across, I these suggestions would tackle the majority. I do come across a few other not so simple ones, though. Here is my current list for which I use in my cleaning scripts (there are some really convoluted ones in there) - could these be picked up or at least flagged for checking somehow? Names on the left are the bad form ones; names on the right are the corrected form:

---
D. & P. Green                                                        : Dave Green & Penny Green
Dr. & Mrs. E & N. J. H. Sturt                                        : Dr. E. Sturt & Mrs. N. J. H. Sturt
Dr. & Mrs. E & N. J. H. Sturt & Dr. A. G. & K. A. Knapp              : Dr. E. Sturt & Mrs. N. J. H. Sturt & Dr. A. G. Knapp & Dr. K. A. Knapp
Dr. & Mrs. E & N. J. H. Sturt & Dr. A. G. Knapp                      : Dr. E. Sturt & Mrs. N. J. H. Sturt & Dr. A. G. Knapp
Dr. & Mrs. E & N. J. H. Sturt & Dr. Judy A. Wilson                   : Dr. E. Sturt & Mrs. N. J. H. Sturt & Dr. Judy A. Wilson
Dr. & Mrs. E & N. J. H. Sturt & Miss F. Abraham                      : Dr. E. Sturt & Mrs. N. J. H. Sturt & Miss F. Abraham
Dr. & Mrs. E & N. J. H. Sturt & Miss F. Abraham & Mr. F. P. Penfold  : Dr. E. Sturt & Mrs. N. J. H. Sturt & Miss F. Abraham & Mr. F. P. Penfold
Dr. & Mrs. E & N. J. H. Sturt & Mr. Howard W. Matcham                : Dr. E. Sturt & Mrs. N. J. H. Sturt & Mr. Howard W. Matcham
Dr. & Mrs. E & N. J. H. Sturt & Mr. J. A. Cole                       : Dr. E. Sturt & Mrs. N. J. H. Sturt & Mr. J. A. Cole
Dr. & Mrs. E & N. J. H. Sturt & Mrs. B. E. Clough & Mrs. J. M. Clark : Dr. E. Sturt & Mrs. N. J. H. Sturt & Mrs. B. E. Clough & Mrs. J. M. Clark
Dr. & Mrs. E & N. J. H. Sturt et al                                  : Dr. E. Sturt & Mrs. N. J. H. Sturt
Dr. & Mrs. E & N. J. H. Sturt, Dr. A. G. & Dr. K. A. Knapp           : Dr. E. Sturt & Mrs. N. J. H. Sturt & Dr. A. G. Knapp & Dr. K. A. Knapp
Dr. A. G. & Dr. K. A. Knapp                                          : Dr. A. G. Knapp & Dr. K. A. Knapp
Dr. A. G. Knapp & Dr. & Mrs. E & N. J. H. Sturt                      : Dr. A. G. Knapp & Dr. E. Sturt & Mrs. N. J. H. Sturt
Dr. A. G. Knapp & M & Berry                                          : Dr. A. G. Knapp & M. Berry
Dr. A. G. Knapp, Dr. O. M. & Dr. M. Hollings                         : Dr. A. G. Knapp & Dr. O. M. Hollings & Dr. M. Hollings
Dr. O. M. & Dr. M. Hollings                                          : Dr. O. M. Hollings & Dr. M. Hollings
Dr. O. M. & Dr. M. Hollings & Dr. A. G. Knapp                        : Dr. O. M. Hollings & Dr. M. Hollings & Dr. A. G. Knapp
J. & N. Luther                                                       : J. Luther & N. Luther
Miss F. Abraham & Dr. & Mrs. E & N. J. H. Sturt & Dr. A. G. Knapp    : Miss F. Abraham & Dr. E. Sturt & Mrs. N. J. H. Sturt & Dr. A. G. Knapp
Mrs. Beryl E. Clough, Dr. A. G. & Dr. K. A. Knapp                    : Mrs. Beryl E. Clough & Dr. A. G. Knapp & Dr. K. A. Knapp
Mrs. E. Bishop & Mr. & Mrs. P. C. & J. Finch                         : Mrs. E. Bishop & Mr P. C. Finch & Mrs. J. Finch
N&ES et al                                                           : N and ES et al
P. H. & C. Jones                                                     : P. H. Jones & C. Jones
Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

charlesroper's Website

4 Reply by Lynn Heeley

  • Lynn Heeley
  • Administrator
  • Offline
  • Registered: 03-07-2008
  • Posts: 345

Re: Import wizard and multiple recorder delimiters

I have logged these issues for future investigation and fixing.
Thanks,
Lynn

5 Reply by ser

  • ser
  • ser
  • Administrator
  • Offline
  • From: Staffordshire Ecological Recor
  • Registered: 24-09-2008
  • Posts: 603

Re: Import wizard and multiple recorder delimiters

I've also found that the import routine doesn't like the default format from Recorder 3, i.e. Surname, Title Initials

e.g. Slawson, Mr G.C.

in the import this creates 3 entries

Slawson, Mr G.C.
Slawson
Mr G.C.

I think it is linked to different processing of collector's name determiner's name, because it doesn't appear to happen where the person isn't the ddeterminer

Craig

Craig Slawson
Staffordshire Ecological Record

ser's Website