* Rearrange Address and Place Parts Plugin

[tatewise]

A protype version of this Plugin is nearing completion.
It should work on both FH V5 and FH V6 with Place records.
It allows Address field and Place field column parts to be re-arranged by copying/moving any column part to any other column part.

It offers shortcut options for the following operations:

• Copy all Address parts to Place parts.
• Copy all Place parts to Address parts.
• Delete all Address parts or all Place parts.
• Shift a chosen subset of Address parts left or right one part.
• Shift a chosen subset of Place parts left or right one part.
• Right justify Address parts to max Column defined in Work with Data > Addresses.
• Right justify Place parts to max Column defined in Work with Data > Places.
• Add or remove [[privacy]] brackets to any Address part that duplicates a Place part.

Are there any other operations that you might need?

Do you think it needs a Result Set of all the changes? It could be VERY large!!!

In FH V6 the resulting Place records will need manual review to merge convergent records, check divergent records, and delete unused records.

At the moment the operations are applied globally to all Address and Place fields, mainly because I cannot think of an easy way to select a subset of fields for the Plugin to work on.

[jimlad68]

I was recently thinking about this and been experimenting with Lua patterns with your Search and Replace, quite possible I think with () and %1, but convoluted. One thing I did note that made it more awkward was if characters like - (minus sign) were within the Place, but that was when moving part of a comma separated field.

Other Features:
<> Result Set, if little extra trouble I would say yes, perhaps optionally, or perhaps to a text/csv file if too much for FH.

<> Add a column (comma separated field) to beginning or end or anywhere in-between.

<> Delete a single column (comma separated field) anywhere.

At the moment the operations are applied globally to all Address and Place fields, mainly because I cannot think of an easy way to select a subset of fields for the Plugin to work on.

It might be long winded for the user but your usual ask for confirmation at each change, but possibly that would not work if a global Lua type change is being used. Possibly filter as in a search.

[tatewise]

I'll think about Result Set options. The problem is very large RS are very slow to open in FH when Plugin closes.

Add/Delete part is handled by the Shift options.

Confirming each change would be tedious on all but smallest Projects, and could lead to inconsistent results. Some sort of filter may offer a solution.

[gsward]

Mike, what sort of "undo" feature might be possible? Step by step or all changes in a session?

[tatewise]

The usual Edit > Undo Plugin Updates per run of Plugin, and similarly for all other edits.

[tatewise]

The prototype Rearrange Address and Place Parts Plugin V0.1 13 Feb 2015 is attached.

The Address & Place Parts on left define the mapping between current and new Parts.
The active Parts are determined by your Work with Data Column settings.
The Options on the right offer shortcuts to some popular mappings.

Nothing actually gets changed until you click the PERFORM REARRANGEMENT button.

Note that the shift left/right Options allow Parts to be inserted and deleted.
e.g.
Change part number on right from 1 to 2 and click Shift right to insert a <blank> Column Part 2 and shift the others right.
Conversely, click Shift left to delete Column Part 2 and shift the others left.

Rearrange address and place parts.png (53.21 KiB) Viewed 12719 times

[jimlad68]

Mike, sorry only time for a quick test and quicker response before I rush off again:

<> The source - target seemed to work fine, I'm thinking you can swap more than 1 column at once and blank will empty it.

<> tried the shift places and got a lot of errors (say very approx 1%), inc some columns with things like {M} in them. However I'm not so sure as to what or how it does it, so I might be trying to do something that is not possible.
[edit] if we cam swap all the parts via source / target, does that not shift places, or does it do something different?

Great idea, both for the future and those moving to FHv6

[tatewise]

You can rearrange as many column parts as you like as defined by the Source to Target mapping on left.

With the Shift Places mapping what sort of errors did you get?

BTW: A full Result Set for a large Project could be enormous, so it only lists one example instance of each unique new Address & Place pair. Therefore, to make further changes either use Tools > Work With Data > Addresses/Places or the Records Window > Places tab.

I forgot to explain the New Place name formats used with FH V6 Place Records in some cases.

If several Old Place names converge to one New Place name, then rather than automatically merge them together the Plugin adds an {M...} numbered suffix to the name, and you must perform the Merge whilst reviewing the Place record values.

If one Old Place name diverges to several New Place names, then the Plugin clones each New Place record from the Old Place records and adds an {alpha-numeric} prefix to each New Place name. These need to be reviewed and the cloned values updated where necessary.

The above naming conventions are designed such that when the Place names are sorted the associated records will be listed consecutively.

[jimlad68]

I must say I don't really understand what shift places does other than it seems to be a quick way of adjusting the source/target that could also be done manually, or does it do something else?

Anyway, I can't get it to fail now (I think my previous experimental data was very messed up after a few trial passes), however I did notice that clicking the shift places button more than once altered the source/target each time clicked.

As regards filtering, would it be possible to use a Named List of _PLACes?

As an aside, is it OK to run this on a standard Gedcom, if so it would be a good idea to run it before merging another Gedcom which might help with the lack of selectivity/filtering.

[tatewise]

Yes (as I said) the Shift (and other) buttons are quick shortcuts to adjust the mapping, but it could take 20 clicks instead of 1 to do it via the lefthand droplists: one click to open the droplist and one click to choose the column part, repeated 10 times for all 10 parts.

But if you have NOT increased the Tools > Work With Data max Columns to 10 it is not possible at all, because the higher numbered part droplists are disabled.

The Shift buttons do what you asked for earlier, and insert or delete column parts. Yes, they can be used repeatedly to insert or delete more than one column.

Having used the Shift, the <blank> parts can be replaced with any column part to migrate parts from say Address to Place.

Yes, with FH V6 (but not V5) it would be possible to select Place records directly from the Records Window or a Plugin prompt (Named List is not required), but Addresses cannot be selected.

The Plugin should work on any Gedcom whether part of a Project or not.

I am slowly adding the plugins:help:rearrange_address_and_place_parts:rearrange_address_and_place_parts|> Help & Advice pages.

[jimlad68]

Works fine for my testing:
- I pre tidied up my columns (all aligned)
- ran moving place column parts 1 by one as required
- no problems.

My only suggestion would be to make the "Restore all the default settings" button even more prominent as it is easy to forget this and hence repeat something you might not want to.

I note there is a new undocumented tab "source filter". I tried this and it appears to work OK. In hindsight I could have used this for part of my initial tidyup.


For info, my latest PLACe column structure which would be easy to change in future with this plugin:

N.B. This is updated in my entry "by jimlad68 » 28 Mar 2015 17:43"

This is for "modern" addresses to tie in with automatic geocoding with things like google maps.
To be used if needed in conjunction with a PLACe SOURCE where extra detail upsets auto geocoding OR for simply giving more historic detail about a place.

[1]-"property name/detail/flat number"
[2] house number (separate column to aid sorting)
[3] street
[4] small village if associated with nearby town OR area of town
[5] Town or Large village
[6] county
[7] state
[8] country
[9] possibly for post code (need to test how it works with google maps and similar)
[10] possibly for lat/long as in TMG, but unfortunately not globally used, and would be Standard Gedcom.

Perhaps not a regularly used plugin, but very very useful when needed, including some of the extras like delete all ADDRESS/PLACE parts.

As ever, much appreciated.

[tatewise]

Thanks for the seal of approval. I can see the Plugin being published next week some time.

I added the Filter tab just out of interest, and it became public when David needed the Plugin in Spaces in Places and Addresses (12361), but I had not got around to adding a Help page, so I'm glad it was obvious how to use it.

[brianlummis]

I have been using this Plugin after importing from TMG and to date it has worked well, especially when using the filter. For some reason I have a lot of instances where the place name is duplicated, appearing both in the address fields and the place fields. Would it be possible to enhance the plugin to be able to delete one column. What I envisage is to use the filter to move these duplicates in the address to a separate column, which I believe is already possible, and then delete them in bulk.

[tatewise]

Brian, there are already Plugin options to fix that problem.

Method 1
One method is to use the Add [[privacy]] to duplicate Addresses tick option, which does not remove the duplicate Address parts, but encloses them in [[square brackets]] so they are hidden in Reports in the same way as [[private]] Notes.

Having done that, if you want to delete them all, then use the Search and Replace plugin.
The Search: pattern you need is %[%[.-%]%],? ?
Select LUA Pattern Mode top right, and untick all Basic Filters except Fact Address fields.
After the Search & Replace some Address fields may be left with a trailing , which can be removed by running the Rearrange Address and Place Parts Plugin with default options, because that is its default tidy action.

Method 2
Since you can identify the duplicate Address parts to move them, you can instead immediately delete them.
Say you have set a Source Filter and know all duplicate Address parts are in column 3.
Set the start Part number near top right to 3 and click Shift Addresses left between:.
This will shift the Address parts in the mapping on the left, and remove the original Address 3 column part.
Then PERFORM REARRANGEMENT will complete the job.

[brianlummis]

Thanks Mike. I have used Method 2 as this seems the quickest way to achieve what I was looking for. Most of my duplicates are in column 1 so highlighting them with the filter and then substituting 3 in your example with 1 works a treat. I thought that it must be possible but couldn't get my head around it! I may try Method 1 as well as that way I will be able to learn a bit more about the power of FH.

Brian

[jimlad68]

change to my above entry of 22 Feb 2015 20:45:

Following the introduction of the "Rearrange Address and Place Parts Plugin" (RAPPP) I have again experimented with my PLACe comma separated fields ordering them with the largest field first e.g.

(N.B. This is for "modern" addresses to tie in with automatic geocoding with things like the Map Life Facts FH Plugin and Google Maps.
To be used if needed in conjunction with a PLACe SOURCE records where extra/old detail upsets auto geocoding OR for simply giving more historic detail about a place nand its people.)

[1] country
[2] state
[3] county
[4] Town or Large village
[5] small village if associated with nearby town OR area of town
[6] street
[7] house number or a place name understood by Google Maps (separate column to aid street number sorting)
[8] -" house number if no longer exist/ property name/ detail/ flat number" = extra FH searchable detail, not for Google maps search, might include old no longer existant house numbers.
(you need to put inside -"xxx" any detail that upsets Google Map search, as it ignores this, similar with Map Life Facts FH Plugin but minus sign not needed - this can be added with the LUA pattern mode in the Search and Replace plugin)
[9] post code
[10] possibly for lat/long as in TMG, but unfortunately not globally used, but would be Standard Gedcom.

I have found that within FH this makes it so much easier to find PLACes, especially when using the Place List (e.g. selecting a place for a Fact) which has rather limited search capabilities.

Drawbacks:
- not the standard order, but then it was never defined as far as I can tell for Gedcom, so it might confuse other recipient programs.
- If doing automatic geocoding type searches with things like Google Maps (and hence Map Life Facts Plugin) sometimes does not give as good results which is more important once outside of FH.

So, to get around this for say Map Life Facts Plugin:
- Run RAPPP to get the fileds in the best order for automatic geocoding searching.
e.g.
[1] house number or a place name understood by Google Maps
[2] street
[3] small village if associated with nearby town OR area of town
[4] Town or Large village
[5] county
[6] state
[7] country
[8] post code
[9] -" property name/detail/flat number" not forgetting the -""
[10] possibly for lat/long as in TMG, but unfortunately not globally used, but would be Standard Gedcom.

- Then perform Map Life Facts Plugin, (or export as Gedcom etc etc)
- when all complete, simply rerun RAPPP again putting the fields in the order you wish for FH use.

Try it on a copied project, only takes a few minutes. You will be amazed at how much faster it is to find PLACes via the Place List.

[davidm_uk]

To be used if needed in conjunction with a PLACe SOURCE records where extra/old detail upsets auto geocoding OR for simply giving more historic detail about a place nand its people.)

Following on from my post: To Move Address into Place Record - or Not (12946) and having read all the suggested (and many other) posts, I'm inclining towards moving all my address details into the PLACE fields along the lines described in the post above by jimlad68, so that I can then attach photos of the places to the PLACE record.

For me this feels better than creating a SOURCE record with a photo attached and then having to link this SOURCE to every FACT containing that place.

However I'm struggling to understand (find out) what is meant by "PLACe SOURCE records" in Jim's post. As far as I can see you can't attach a SOURCE to a PLACE.

Perhaps someone could explain?

[jimlad68]

re

However I'm struggling to understand (find out) what is meant by "PLACe SOURCE records" in Jim's post

In this instance I use a Source for a fact, that tells me about a place. So it is a standard source that gives info about a place. Without rummaging through my reasons I think it was around geocoding and portability of data, in that:

- geocoding - requires modern Place (addresses), so that would loose historic data etc, so where to put it?

- ideally I would like to put all the details about a place in the new FH place record, so that in theory whenever the place is mentioned you could get that info too. BUT, that is not very portable and as yet, I don't think very accessible in certain FH reports.

So I went along the route of creating a source with the place name (or similar) containing details of the place (could be old street name, changed numbers, history, occupants, pictures etc etc) and add this source to the relevant fact where the place is added. This is not ideal, as it means connecting a source whenever a place in recorded (rather than the automatic new FH place text detail), however a source is not required for every place, and one source might cover a whole street or even locality. The main problem is remembering to add the source.

The wonder and flexibility of sources.

[davidm_uk]

WOW, that was quick! Thanks for the explanation Jim.

I'll now make a copy of my GED file and have a play with Mike's RAPP Plugin to reorganise my address and place data.