* Possible error in Clean up Surname Capitalisation Plugin
- johnhanson
- Diamond
- Posts: 80
- Joined: 27 Nov 2002 16:50
- Family Historian: V7
- Contact:
Possible error in Clean up Surname Capitalisation Plugin
I am trying to run Jane Taubman's Clean up Surname Capitalisation plugin
When I started family history over 40 years ago Capitalising surnames was the norm.
People keep saying to me that the FH help advises not to - changing 40 year habits is not easy!
So I thought I would try Jane's plugin to change mine (in a copy of the main database.
I downloaded and ran it and it produced this result
I am reluctant to continue - I suspect that the full list is there but can't be certain
There are 41,000 plus people in the database
When I started family history over 40 years ago Capitalising surnames was the norm.
People keep saying to me that the FH help advises not to - changing 40 year habits is not easy!
So I thought I would try Jane's plugin to change mine (in a copy of the main database.
I downloaded and ran it and it produced this result
I am reluctant to continue - I suspect that the full list is there but can't be certain
There are 41,000 plus people in the database
Last edited by tatewise on 24 Apr 2024 09:51, edited 1 time in total.
Reason: Corrected [img] code
Reason: Corrected [img] code
John Hanson FSG
Researcher, the Halsted Trust
Researcher, the Halsted Trust
Re: Possible error in Clean up Surname Capitalisation Plugin
As it's on a copy I think I'd give it a go. You can always delete the project and copy again if it goes wrong.
John Elvin
- Mark1834
- Megastar
- Posts: 2534
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: Possible error in Clean up Surname Capitalisation Plugin
That is how it is designed to work. The plugin description in the Plugins Window tells you that it "allows over-ride at runtime by prompting the converted surnames for confirmation."
Options for this type of simple plugin are rather limited:
A simpler option for the user to implement if they have a large database is to run a simple query listing full name, surname and Record ID on the project before and after conversion. Output the two files to spreadsheets and copy one into the other, collating entries on Record ID.
Options for this type of simple plugin are rather limited:
- Make all changes unconditionally. Quick and simple, but names can be complex, so not all changes would necessarily be correct on surnames that are other than a single word.
- Present the full list of new surnames for user review and manual changing of individual ones before committing the change. That's the option chosen here.
- Prompt all changes one by one. Thorough, but slow and impractical.
- Output the proposed changes to the Query Window where it would be easier to review or export, say to a spreadsheet, but there is no mechanism for correcting any of the proposed changes.
A simpler option for the user to implement if they have a large database is to run a simple query listing full name, surname and Record ID on the project before and after conversion. Output the two files to spreadsheets and copy one into the other, collating entries on Record ID.
Mark Draper
Re: Possible error in Clean up Surname Capitalisation Plugin
Mark, I think his point is that the list is truncated half way through a name (N.B. the scroll bar is at the bottom).
Edit: I suspect it's gone over the maximum length of a text box.
Edit: I suspect it's gone over the maximum length of a text box.
John Elvin
- Mark1834
- Megastar
- Posts: 2534
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: Possible error in Clean up Surname Capitalisation Plugin
Ah - ok. Assuming the plugin does actually change the listed names, the solution would be to delete the truncated last entry, then repeat with successive plugin runs until all the names are converted.
Ideally, the plugin should identify why it has failed, but this type of simple plugin doesn't necessarily have a full suite of error handling functions.
Ideally, the plugin should identify why it has failed, but this type of simple plugin doesn't necessarily have a full suite of error handling functions.
Mark Draper
- johnhanson
- Diamond
- Posts: 80
- Joined: 27 Nov 2002 16:50
- Family Historian: V7
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Mark
It isn't erroring
I suspect that it just isn't displaying all of the material but I would rather make sure than run it multiple times
I would think something like 7 or 8 times bearing in mind where in the alphabet it has stopped
As an ex programmer I never like things that don't display properly
It isn't erroring
I suspect that it just isn't displaying all of the material but I would rather make sure than run it multiple times
I would think something like 7 or 8 times bearing in mind where in the alphabet it has stopped
As an ex programmer I never like things that don't display properly
John Hanson FSG
Researcher, the Halsted Trust
Researcher, the Halsted Trust
- Mark1834
- Megastar
- Posts: 2534
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: Possible error in Clean up Surname Capitalisation Plugin
Another possibility is that it will convert all the names in one pass, but is only capable of displaying some of them. In that case, there is no option to review the rest of the names. I'm away from the desk until this evening so can't review the code to see what it is doing, but I agree that it shouldn't leave the user guessing as to what is happening.
It's Jane's plugin, so I'll expect she'll comment when she's back on the forum.
It's Jane's plugin, so I'll expect she'll comment when she's back on the forum.
Mark Draper
- tatewise
- Megastar
- Posts: 28488
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
FYI:
The plugin is using iup.GetText(...) which displays a modal dialog to edit multiline text.
The idea is that the user reviews and corrects each surname listed.
However, in Lua the default maxsize is 10240 characters, so I believe the dialog is being truncated.
So it is currently not possible to review and correct beyond the surname Dodgeon.
If maxsize is set to 0 then it will be the current length of text, and then all surnames should be listed.
Although the plugin claims to have been updated for FH V7 it does not set its Type to Standard and does not cope with UTF-8 accented or foreign language characters.
IMO: This plugin needs a minor overhaul to correct these issues.
The plugin is using iup.GetText(...) which displays a modal dialog to edit multiline text.
The idea is that the user reviews and corrects each surname listed.
However, in Lua the default maxsize is 10240 characters, so I believe the dialog is being truncated.
So it is currently not possible to review and correct beyond the surname Dodgeon.
If maxsize is set to 0 then it will be the current length of text, and then all surnames should be listed.
Although the plugin claims to have been updated for FH V7 it does not set its Type to Standard and does not cope with UTF-8 accented or foreign language characters.
IMO: This plugin needs a minor overhaul to correct these issues.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- johnhanson
- Diamond
- Posts: 80
- Joined: 27 Nov 2002 16:50
- Family Historian: V7
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Mike
Thanks for the comments
As this is one of Jane Taubman's how can I get it changed?
Thanks for the comments
As this is one of Jane Taubman's how can I get it changed?
John Hanson FSG
Researcher, the Halsted Trust
Researcher, the Halsted Trust
- tatewise
- Megastar
- Posts: 28488
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
John, you can edit the plugin yourself to make any necessary changes.
One of us could advise what changes to make once you are familiar with the editing process.
Alternatively, you must wait until Jane notices this thread and updates the plugin in the Plugin Store.
One of us could advise what changes to make once you are familiar with the editing process.
Alternatively, you must wait until Jane notices this thread and updates the plugin in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- johnhanson
- Diamond
- Posts: 80
- Joined: 27 Nov 2002 16:50
- Family Historian: V7
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Yep
Should be able to do this if someone tells me what to change
Then Jane can catch up later
Should be able to do this if someone tells me what to change
Then Jane can catch up later
John Hanson FSG
Researcher, the Halsted Trust
Researcher, the Halsted Trust
- tatewise
- Megastar
- Posts: 28488
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Ok, the following two changes should be sufficient, unless your Surnames use foreign language characters.
Near line number 16 add three lines after iup.SetGlobal("CUSTOMQUITMESSAGE","YES") to cope with UTF-8
Near line number 85 change how iup.GetText(...) is called to specify its max size as the length of the Surname list
BTW: I tried setting the max size parameter to 0 as defined in the IUP documentation to use the current length of text but that crashes FH without any error messages.
Near line number 16 add three lines after iup.SetGlobal("CUSTOMQUITMESSAGE","YES") to cope with UTF-8
Code: Select all
require("iuplua");
iup.SetGlobal("CUSTOMQUITMESSAGE","YES")
iup.SetGlobal("UTF8MODE","YES")
iup.SetGlobal("UTF8MODE_FILE","NO")
fhSetStringEncoding("UTF-8")
Code: Select all
local strNames = table.concat(tblSurname,'\n')
strList = iup.GetText('Please check the Case on the Surnames Below', strNames, #strNames+1)
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- Mark1834
- Megastar
- Posts: 2534
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: Possible error in Clean up Surname Capitalisation Plugin
Minor clarification, which I’m sure I’ve made before…
Plugins need the additional UTF functions to process non-Latin characters, not “accented or foreign ones”. It’s not necessary for the common Western European diacritics, and what’s “foreign” viewed from a desk in Devon or Cheshire may not be foreign to the large number of readers who are from a different Parish!
Plugins need the additional UTF functions to process non-Latin characters, not “accented or foreign ones”. It’s not necessary for the common Western European diacritics, and what’s “foreign” viewed from a desk in Devon or Cheshire may not be foreign to the large number of readers who are from a different Parish!
Mark Draper
- tatewise
- Megastar
- Posts: 28488
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Just to be clear, the extra symbols and diacritic letters supported by ANSI coding are only:
€ ‚ ƒ „ … † ‡ ˆ ‰ ‹ Æ æ Œ œ ‘ ’ “ ” • – — ™ › ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ × ÷ Þ þ ß Ð ð nbsp
À à Á á Â â Ã ã Ä ä Å å Ç ç È è É é Ê ê Ë ë Ì ì Í í Î î Ï ï Ñ ñ Ò ò Ó ó Ô ô Õ õ Ö ö Ø ø Š š Ù ù Ú ú Û û Ü ü Ý ý Ÿ ÿ Ž ž
The diacritic letters not supported by ANSI and only allowed in UTF-8 coding include (but there are more):
Ā ā Ă ă Ą ą Ć ć Ĉ ĉ Ċ ċ Č č Ď ď Đ đ Ē ē Ĕ ĕ Ė ė Ę ę Ě ě Ĝ ĝ Ğ ğ ġ ģ Ĥ ĥ Ħ ħ Ĩ ĭ Į į IJ ij Ĵ Ķ ķ ĸ Ĺ ĺ Ļ ļ ľ Ŀ Ł ł
Ń ń Ņ ņ ʼn Ŋ ŋ Ō ō Ŏ ŏ Ő ő Ŕ ŕ Ř ř Ś ś Ŝ ŝ ş Š ţ Ţ Ť ť Ũ ũ Ū ū Ŭ ŭ Ů ů Ű Ų ų Ŵ ź ż Ž ž
I leave it up to the reader to decide whether the ANSI set covers the common Western European diacritics.
€ ‚ ƒ „ … † ‡ ˆ ‰ ‹ Æ æ Œ œ ‘ ’ “ ” • – — ™ › ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ® ¯ ° ± ² ³ ´ µ ¶ · ¸ ¹ º » ¼ ½ ¾ ¿ × ÷ Þ þ ß Ð ð nbsp
À à Á á Â â Ã ã Ä ä Å å Ç ç È è É é Ê ê Ë ë Ì ì Í í Î î Ï ï Ñ ñ Ò ò Ó ó Ô ô Õ õ Ö ö Ø ø Š š Ù ù Ú ú Û û Ü ü Ý ý Ÿ ÿ Ž ž
The diacritic letters not supported by ANSI and only allowed in UTF-8 coding include (but there are more):
Ā ā Ă ă Ą ą Ć ć Ĉ ĉ Ċ ċ Č č Ď ď Đ đ Ē ē Ĕ ĕ Ė ė Ę ę Ě ě Ĝ ĝ Ğ ğ ġ ģ Ĥ ĥ Ħ ħ Ĩ ĭ Į į IJ ij Ĵ Ķ ķ ĸ Ĺ ĺ Ļ ļ ľ Ŀ Ł ł
Ń ń Ņ ņ ʼn Ŋ ŋ Ō ō Ŏ ŏ Ő ő Ŕ ŕ Ř ř Ś ś Ŝ ŝ ş Š ţ Ţ Ť ť Ũ ũ Ū ū Ŭ ŭ Ů ů Ű Ų ų Ŵ ź ż Ž ž
I leave it up to the reader to decide whether the ANSI set covers the common Western European diacritics.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- Mark1834
- Megastar
- Posts: 2534
- Joined: 27 Oct 2017 19:33
- Family Historian: V7
- Location: South Cheshire, UK
Re: Possible error in Clean up Surname Capitalisation Plugin
True, we should probably settle on “non-Latin” as the preferred term, as it has a recognised definition. “Common” is a bit subjective, and “foreign” is meaningless when the author and reader may be of different nationalities.
Mark Draper
- johnhanson
- Diamond
- Posts: 80
- Joined: 27 Nov 2002 16:50
- Family Historian: V7
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Mike
Thanks that worked nicely - now to fix the rest which I can do easily
The only thing that is missing from the resulting list is the User ID
Something that a number of these simple plugins seem to miss in the early versions
Is it easy to add to the result output?
Thanks that worked nicely - now to fix the rest which I can do easily
The only thing that is missing from the resulting list is the User ID
Something that a number of these simple plugins seem to miss in the early versions
Is it easy to add to the result output?
John Hanson FSG
Researcher, the Halsted Trust
Researcher, the Halsted Trust
- tatewise
- Megastar
- Posts: 28488
- Joined: 25 May 2010 11:00
- Family Historian: V7
- Location: Torbay, Devon, UK
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
To add a Record Id to the final Result Set ...
About line 95 add a table variable to hold the Record Id:
About line 115 save the Record Id in the table:
About line 128 display Record Id in the Result Set:
About line 95 add a table variable to hold the Record Id:
Code: Select all
tblRecord = {}
tblRec_Id = {}
Code: Select all
table.insert(tblRecord,pi:Clone())
table.insert(tblRec_Id,fhGetRecordId(pi))
Code: Select all
fhOutputResultSetColumn("Individual", "item", tblRecord, #tblRecord, 180, "align_left")
fhOutputResultSetColumn("Rec Id", "integer", tblRec_Id, #tblRecord, 40, "align_mid")
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
- johnhanson
- Diamond
- Posts: 80
- Joined: 27 Nov 2002 16:50
- Family Historian: V7
- Contact:
Re: Possible error in Clean up Surname Capitalisation Plugin
Mike
Worked like a dream
Thank you so much
Worked like a dream
Thank you so much
John Hanson FSG
Researcher, the Halsted Trust
Researcher, the Halsted Trust