* Search and Replace Plugin (FH7)

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Search and Replace Plugin (FH7)

Post by David Potter » 24 Jan 2021 13:33

Help please...

I'm mostly using AS7 to create my Census entries into FH7 and one very useful AS7 feature is the Place/Address Tidy option whereby additional spaces and comma's are removed before being sent to FH7.

However, I make good use and wish to keep double comma's in my Place records in order to position parts of the Place name into specific Columns. EG, Ramsgate, Nr Margate, Kent,, England.

But I would like to remove such double comma's from the Text from Source templated data from the Census records.

I have tried using the Search and Replace Plugin provided by Mike Tate but I cannot get it to ignore Place data. I need some guidance in which settings would allow the Plugin to update Text from Source info only in the TFS table shown in the attached image embedded into the attached Word Doc due to image size restrictions.

Thank You.
Attachments
Doc1.docx
(39.52 KiB) Downloaded 89 times

User avatar
tatewise
Megastar
Posts: 27087
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin (FH7)

Post by tatewise » 24 Jan 2021 13:58

In future, the easiest way to avoid such additional commas is to review & edit the Auto Text table in AS before saving to FH.
You should always try to make that text match the original document as a faithful transcript.

As shown in your attachment, those additional commas only occur in Source records in Text From Source fields.

So in the plugin set the Search Scope to Source Records (SOUR) and in the Basic Filters tick only Text From Source fields.

In the Basic Filters did you leave Place fields ticked? That might explain why those fields got changed.

David, can you please explain why the above settings were so difficult to work out.
If possible I would like to improve the plugin to make it easier to choose appropriate settings.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
LornaCraig
Megastar
Posts: 2996
Joined: 11 Jan 2005 17:36
Family Historian: V7
Location: Oxfordshire, UK

Re: Search and Replace Plugin (FH7)

Post by LornaCraig » 24 Jan 2021 14:00

David, if you want to have a true record of what was in the original text from source you really need to do a lot more than just remove the extra comma. I doubt whether the original text really said "Wraxhall, Nr Nailsea, Somerset, England". It probably just said "Wraxhall, Somerset". Such edits are best done in AS before you save to FH. The double comma after Somerset is a relatively minor issue.
Lorna

User avatar
NickWalker
Megastar
Posts: 2401
Joined: 02 Jan 2004 17:39
Family Historian: V7
Location: Lancashire, UK
Contact:

Re: Search and Replace Plugin (FH7)

Post by NickWalker » 24 Jan 2021 14:02

If you were using the Place/Address tidy feature then the example in your attached document wouldn't have appeared like that. Or do you mean you have a number of entries made before you turned that option on? AS doesn't remove commas or spaces from place records, only from the source text if you use Auto-text (Apart from source template fields which is a bug you reported and will be fixed in the next version of AS).
Nick Walker
Ancestral Sources Developer

https://fhug.org.uk/kb/kb-article/ancestral-sources/

avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin (FH7)

Post by David Potter » 24 Jan 2021 15:14

Thank you Mike very much appreciated for that setup info. What you sent me was very close to what I had been testing with. But for some reason when I ran it previously it brought into play Place details? I think I may have gotten confused with the Extra Filters, thinking something needed to be set there also, obviously Text From Source fields needed to be set there or on the Basic Filters page in anycase. And another confusion was that the Select Records showed 252 (Sour) records are selected just to the right of that Button. But in actual fact when you click the Select Records button no records appear in the Right Hand selection window. So confusion I guess as to what is needed to update those TFS fields only. I'll get there one day... It has worked now, Thank you.

@All - Yes I'm aware of the need to make the necessary tweaks in AS before updating FH, this was a relatively new feature in AS7 I only started using yesterday. I also note AS7 has a Shorten Where Born function requested by Lorna. I will experiment with this shortly. I'll add this 'adjustment' to clean out un-necessary Where born info to my Clean Up task list once I have got my head around how best to apply this practice going forward.

These Census records were created before those two functions became known to me so this is part of a Catch Up/Clean Up exercise. I know how important it is to be True to the Source. I guess I just didn't know how to apply this for Place data without ending up with many 'partial' Place Names. But then Standardized Place Name comes in to play, another feature I still need to get up and running.

@Lorna, Yes I understand this Lorna and totally agree, but this was a Do-Over project which rather than start from nothing I used a cleaned down Project from FH6 which contained only People and Places nothing else. So that bad habit of Place data came from a legacy project.

@Nick, Yes Nick these records came along before I turned that AS7 Feature on.

Lorna or Mike can I ask either of you for a Sample/Example of how you, for Census entries handle the Where Born place details. I still have 1851 and 1841 Census records to enter via AS7 into FH7. Some useful guidance on best practice and in a practical manner by example would help me tremedously. And reduce my Clean Up Task.

I'm thinking what you do for a Census containing a family of 5 Individuals who were all born in different locations is this:

Where Born
Somerset, Camerton
Gloucestershire, Ruardean
Worcestershire, Woodmancote
Somerset, Nailsea
Somerset, Wraxhall

Creates 5 'Short' Place records as above in FH either manually or via AS.
But then you would use the Standard Place field to enter for the first entry:
Camerton, Nr Carlingcott, Somerset, England

Is that the basis of what you do?

I appreciate you might frown on the use of the Nr somewhere, but it helps me place Places on the FH Mapping function where there are many occurences of that Place in a Country.


Thank you All

David.

avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin (FH7)

Post by David Potter » 24 Jan 2021 15:28

Hi All

@ Mike and Lorna - Is this what you do for best practice for Place Names associated with Census data...? See attached.

Thank you
Attachments
Doc1.docx
(25.19 KiB) Downloaded 110 times

User avatar
tatewise
Megastar
Posts: 27087
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin (FH7)

Post by tatewise » 24 Jan 2021 15:42

Search and Replace plugin
If it said 252 (Sour) records are selected then in some recent run of the plugin you have clicked the Select Records button, which the plugin remembers. To clear that either choose another Search Scope option and then reset to Source Records (SOUR) or as it says in the Help & Advice use Select Records and Cancel to restore full set.

Place names
No, you have got confused about our advice.
In the AS top Census Grid always enter the full Place name as it is recorded in your Project data.
That is the Place field value that will be used in Birth facts and must match you Project data.
If you enter abbreviated Place names then you will get all manner of extraneous Place records created as shown by your screenshot.

The Standardized field is only intended for mapping historic Place names to modern 'standardized' Place names to assist with geocoding.

BTW: The same applies to Occupation. Choose those that are used in your Project data.

It is only in the Auto-Text field that is edited to form a faithful transcript of the document.
That affects Names, Occupation, Where Born, etc, etc, and often involves abbreviating & reversing the place name parts.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
LornaCraig
Megastar
Posts: 2996
Joined: 11 Jan 2005 17:36
Family Historian: V7
Location: Oxfordshire, UK

Re: Search and Replace Plugin (FH7)

Post by LornaCraig » 24 Jan 2021 15:44

If the place already exists in FH then when you start to type it in the Where Born column of the main census grid in AS it will auto-complete, in full form. But note that if you already have a place called Camerton, Somerset, England, you need to type it in like that in AS. It won't know that Somerset, Camerton is the same place!

If the place does not already exist in FH then AS will create it for you, but you must enter the full version of the name in the main census grid. Then the autotext will show "Camerton, Somerset, England" or (if you use the new Shorten Where Born option) "Camerton, Somerset". It's up to you if you then want to reverse the order to display "Somerset, Camerton" in the Text.

Note that the new Shorten option will display the first two place parts, but in your case you will still need to make some adjustments where your second place part is one of the "near xxx" ones. The two parts recorded in a UK census are normally just village (or parish or town) and county.
Lorna

avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin (FH7)

Post by David Potter » 24 Jan 2021 15:51

Thank you both.

Bingo - it has clicked now. So I'm not so far away from doing this right and only need to realign the Text from Source Where Born parts in FH to match the Source the Record.

Thank you all for your patience and assistance.

David

avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin (FH7)

Post by David Potter » 25 Jan 2021 11:04

@Lorna
Hi Lorna, may I please ask how you structure your Place details in order to make use of the AS7 Shorten Where Born you requested Nick to code? Practically all my UK Census records have the County followed by Town, Village, Hamlet.

This would suggest if the Place is Shortened up to the Second Comma, then the Place details need to be simarly structured to give County, Town, etc.

OR does this Shorten purely leave a simplified Place value which then requires a further manual edit. IE, only 2 values to edit or swap around as opposed to 4 or 5.

Many thanks.

David

User avatar
tatewise
Megastar
Posts: 27087
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Search and Replace Plugin (FH7)

Post by tatewise » 25 Jan 2021 11:32

David, why not experiment and see? Perhaps in the Family Historian Sample Project.
Just don't Save to FH when done, simply close AS and agree to the warning that data will be lost.
That is the way I discover how many of the FH and AS features work, along with reading the Help :roll:
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

User avatar
LornaCraig
Megastar
Posts: 2996
Joined: 11 Jan 2005 17:36
Family Historian: V7
Location: Oxfordshire, UK

Re: Search and Replace Plugin (FH7)

Post by LornaCraig » 25 Jan 2021 12:44

OR does this Shorten purely leave a simplified Place value which then requires a further manual edit. IE, only 2 values to edit or swap around as opposed to 4 or 5.
Yes. I noticed that AS already had an option to match the FH 'tidy' option (removing redundant commas in display of Place names) and suggested that it would be useful to have another option to match the existing FH 'medium' option, which displays text up to the second comma only.

Obviously the effect of this option is dependent on the user’s configuration of places and addresses and it wouldn’t help everyone. Some people may need more than two parts to be retained, so 'medium' would truncate too much. In your case where you have 'near xxx' as the second part it will truncate too much. But a typical UK census normally gives only a two-part birth place, the second part being the county. So ‘England’, ‘Scotland’ etc always has to be removed from the autotext. In cases where the actual text shows the county first (no more than half the cases, in my experience) you still need to swap the first two parts round manually in the text from source.
Lorna

User avatar
NickWalker
Megastar
Posts: 2401
Joined: 02 Jan 2004 17:39
Family Historian: V7
Location: Lancashire, UK
Contact:

Re: Search and Replace Plugin (FH7)

Post by NickWalker » 25 Jan 2021 13:22

The 'tidy' feature has been there since 2014. The 'shorten' feature hasn't been released yet and is only in the test version that a small number of users are testing (but should be released soon). All these auto-text features are attempting to reduce the amount of work you need to do to create the transcription of the source record. So if you don't use the shorten option you'll have to manually remove part of the place sometimes and if you do shorten you'll need to add some extra text sometimes. You would just need to decide which of those is likely to save you the most work. The way places are described on censuses are often very different to the standardised Place field that you use in facts, using abbreviations, etc. It's up to you as to whether you think it really matters whether the transcript says e.g. G'Hall or Grappenhall or even ditto (referring to a row above) regardless of what the actual census record says as all obviously refer to the same place.
Nick Walker
Ancestral Sources Developer

https://fhug.org.uk/kb/kb-article/ancestral-sources/

avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin (FH7)

Post by David Potter » 25 Jan 2021 20:09

Hi Everyone, sorry; for some reason I did not receive any notification of these new posts.

Thank you - I'm going through your comments now.

David.

avatar
David Potter
Megastar
Posts: 957
Joined: 22 Jun 2016 15:54
Family Historian: V7
Location: United Kingdom

Re: Search and Replace Plugin (FH7)

Post by David Potter » 25 Jan 2021 20:21

Thank you All, that helped a lot.

Mike I didn't quite get the reason for the Rolling Eyes gesture. The help tells you first and foremost how and not why. I'm just trying to understand how others do things, to glean best practice and avoid issues going forward.

I've spent a lot of time experimenting, only to find out a long time after the route chosen had flaws.

Thank you all for your continued support.

Post Reply