1 Topic by HarryClarke

  • HarryClarke
  • Veteran
  • Offline
  • From: Surrey
  • Registered: 11-10-2011
  • Posts: 99

Topic: Name Matches - big frustration

The new recorder name matching in version 6.30.0.291 of Recorder 6 is extremly frustrating. After edits to the source data finanly managed to import some records from iRecord.

1) Recorder 6 can now say there are multiple possible entries for a name. However, there is only one extact match, and various other fuzzy matches. If there is an exact match, then Recorder 6 should use that. I am not interested that there are other vague matches. I don't know and I don't care that "J Smith" may be or maybe not "John Smith" when I already have a "John Smith" entry. Unless I know the person, and they have a rare name, I just don't whether they are the same person or two different people.
My rule, is that if there is not an exact match, create a new person. I tried in the past to clean up the database to remove duplicates, and it was too much hard work for too little reward.

2) People with double barrel surnames are a pain to enter into Recorder 6, as there seems to be no format that Recorder 6 understands. the real issue is that Recorder 6 can treat comma to mean either surname, forename or recorder 1 and record 2. The standard should be comma means "surname, forename". I usually find out there is a problem, when Recorder 6 wants to create a person with just their forename.

Harry Clarke
Surrey County Butterfly Recorder

2 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,637

Re: Name Matches - big frustration

The complaint was that Recorder was too free in the way it would match or automatically create a John Smith or just match a John Smith to an existing John Smith  when there were several other possiblibilties which it could be eg J.Smith, Mr Smith etc. This was leading to matches being made which were not correct, without the user being given the opportunity to check.  With the new matching it is possible to get a lot more information about the recorders in the system, which can help with deciding who the correct one is, or if a new Recorder is required. There are multiple views on this issue.  Some believeing that the accurate allocation of a person to a record is as important as knowing the date recorded. These users do not want the system making spurious choices for them.  Others just recording the name because it is there and not caring if the person could ever be indentified from the information. This  is perhaps something which needs to be debated via Trello. From a systems point of view it is hard to justify allowing the system  to make decisions for users when there is insufficient or conflicting information. Name matching has as far as possble been brought into line with species matching in that where there appear to be multiple possibilities then the onus is on the user to make the choice.

I agree with your point on hypenated names. The parser doesn't handle these very well. Various solutions were propsed in the JNCC days, but all have drawbacks. I will go back through the work done on this and see if we can now see a way forward.

Mike Weideli

MikeWeideli's Website

3 Reply by nmrs

  • nmrs
  • Veteran
  • Offline
  • From: London
  • Registered: 22-11-2006
  • Posts: 387

Re: Name Matches - big frustration

Hi Mike,

Is there a possibility this can be looked at again please? I can spend over an hour in large imports matching recorder names I know I have done many times before without the possibility of committing the match as the button is greyed out. I get why the current process was implemented but this is wasting a tremendous amount of my time. I'm just about to start on another import with ~500 recorder names/aliases to match for example - there has to be a better way?

Les Evans-Hill
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme

nmrs's Website

4 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,637

Re: Name Matches - big frustration

Hi

Can you give me some idea what you think should happen ?  Automatic matching can be made easier, but this will mean that the system will match/create names  in situations where there is really insufficient information to do so correctly. Can you give me some examples where you would consider the system should match/create names automatically and it isn't doing so.

Are you using the search facility in the 'Notes' column to help with the matching ? This should provide suggestions from which you can select the appropriate  match.

Mike Weideli

MikeWeideli's Website

5 Reply by nmrs

  • nmrs
  • Veteran
  • Offline
  • From: London
  • Registered: 22-11-2006
  • Posts: 387

Re: Name Matches - big frustration

Hi Mike, just got sight of this sorry.

I am as I write matching about 300 names where there is between 1 and 8 possible matches in the Notes column. I'm not interested in matching every single alias to a single name (J. Smith could possibly be John Smith, Jane Smith, J.E.Smith etc.), it would take me too long to manage as moth recording generates far too many aliases that cannot reliably matched. I basically need the same matching process as was implemented before the current 'possible match' process, i.e. if the match is exact, that'll do nicely, match it, and if there is no match I'll just create a new entries/alias.

I know this isn't ideal or good practice, but as I mentioned earlier, it takes far too long, 1-2 hours to go through all the unmatched names in some of the larger datasets, look for a match and if exists, match it. Previously it took about a minute!

Apologies Mike, but this is extremely frustrating and I'm spending hours of my time sitting in front of a screen just doing these matches and while I'm doing these I cannot tend to other data management matters. It just isn't good use of my time.

Les Evans-Hill
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme

nmrs's Website

6 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,637

Re: Name Matches - big frustration

Hi

The crticism of the previous approach was probably justified in that R6 was matching in situations where there was insufficient information to do so with any level of acurracy. Some users feel very strongly that names should be controlled as carefully as species allocations or spatial information and were unable to use the automatic matching because of the amount of checking necessary afterwards.

Ideally the steering group needs to get some consensus on this.

If you change the Surveys to Temporary (see the Licence Tab) then you can import the names in an unparsed format (ie as they appear in the coulumn). Have a look at this option and see if it helps.

Mike Weideli

MikeWeideli's Website

7 Reply by nmrs

  • nmrs
  • Veteran
  • Offline
  • From: London
  • Registered: 22-11-2006
  • Posts: 387

Re: Name Matches - big frustration

Mike,

Completely happy with other users criticism of the previous approach and I fully appreciate that some users want more control over such data. For the NMRS, because of the overwhelming number of aliases for a single recorder, it just isn't practical for me to match the names where it is impossible for me to do so with any accuracy, there's 1,000s of moth recorders out there so you can imagine the size of the task.

If this can be discussed at the next Steering Group I'd be delighted. I'll give the process you describe a go to see if this helps. Once again, apologies for being a pain; however, trying to keep tracks of moth recorder names would be a full time job in itself if I were to require the same level of control of recorder names as some users rightfully desire.

Les Evans-Hill
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme

nmrs's Website

8 Reply by nmrs

  • nmrs
  • Veteran
  • Offline
  • From: London
  • Registered: 22-11-2006
  • Posts: 387

Re: Name Matches - big frustration

Hi Mike, unless I'm missing something, I don't have a Licence Tab or any kind of option in my install of R6 - 6.30.0.291 C3/4V.

Les Evans-Hill
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme

nmrs's Website

9 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,637

Re: Name Matches - big frustration

Hi

The Licence tab is on the Survey. Making the Survey a temporary one will allow you to import names without parsing. The downside of this is that data in temporary Survey can't be exported to other user or the NBN. If this is an issue for you then it is best not to use this

I am going to try and make some changes to help with this problem in the 6.31 release  which is in course of preparation.

Mike Weideli

MikeWeideli's Website

10 Reply by MikeWeideli

  • MikeWeideli
  • Administrator
  • Offline
  • From: Littlefield Consultancy
  • Registered: 05-07-2006
  • Posts: 2,637

Re: Name Matches - big frustration

Hi

Please find attached a revised Usp which I think should help with this issue. If you can run this in Management studio it will change the Matching of Names back to what is was in previous versions. Let me know if this helps and if so it will be incorporated as an option in 6:31.

Post's attachments

RevisedMatching.sql 2.99 kb, 3 downloads since 2021-02-09 

You don't have the permssions to download the attachments of this post.
Mike Weideli

MikeWeideli's Website

11 Reply by nmrs

  • nmrs
  • Veteran
  • Offline
  • From: London
  • Registered: 22-11-2006
  • Posts: 387

Re: Name Matches - big frustration

Mike , many thanks I'll test this on my development database. As we plan to export to the NBN (eventually!), on your advise I didn't go down the temporary survey route.

Les Evans-Hill
Senior Data Officer
Butterfly Conservation, Butterflies for the New Millennium and National Moth Recording Scheme

nmrs's Website