* Rearrange Address and Place Parts 1.6 Errors Out

[BakerJL75]

I suspect maybe something is wonky with my data. It won't even finish if all I'm doing is adding a <blank> to column 2 which already contains no data. I'm not sure I even need to do that, but https://pluginstore.family-historian.co ... lace_field says The first step is to insert a <blank> into the Place parts where the Address part will go.

I get the error message below and it won't continue even if I say Continue

[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] attempt to index a nil value (field '?')
stack traceback:
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in function 'DoRearrangeParts'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in upvalue 'doPerform'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4200: in function <[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4200>
(...tail calls...)
[C]: in function 'iuplua.MainLoop'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in field 'ShowDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4251: in function 'GUI_MainDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4267: in main chunk

Here is the layout if it helps

Screenshot 2021-05-20 103429.png (138.92 KiB) Viewed 4766 times

Columns 1 and 2 of my data are either <BLANK> or empty. My end goal is to move Address 1 into place 1 and Address 2 into place 2. When that err'd out and I checked the FAQ I thought maybe column 2 needed to say <BLANK> although I don't remember that from the first time I used this. But that didn't run either. I did run Verify on my data and it found nothing.

Thanks for any ideas.

[tatewise]

Jackie, I am investigating that error but have family round for dinner this evening so will have to wait until tomorrow.
There is something unusual about your data because the odd Place structures I've tried don't fail.
I will also explain the other issues tomorrow.

[BakerJL75]

No hurry. I have plenty of other stuff to do. Enjoy your family.

[tatewise]

I have added a diagnostic report to try and discover what is causing the error.
Please download and install the attached Rearrange Address and Place Parts plugin Version 1.6.2 Date 21 May 2021.

Run this plugin exactly as before and it should produce an Error Report Data popup message box.
Please post a copy of that message here.
When you click OK the plugin will terminate.

Assuming they don't say ? please provide details of either the oldPlac &/or newPlac names listed from your Records Window on the Places tab of Place records.

[BakerJL75]

I don't think this will help much. It did exactly what it did the first time. I'll post the error message below, but looks the same to me. I glanced at the code, and I don't think your error checking ran. My message said Error! and if I understood the code your new message should have said Error Report Data.

[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] attempt to concatenate a nil value (local 'oldAddr')
stack traceback:
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in function 'DoRearrangeParts'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in upvalue 'doPerform'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4218: in function <[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4218>
(...tail calls...)
[C]: in function 'iuplua.MainLoop'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in field 'ShowDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4269: in function 'GUI_MainDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4285: in main chunk

Nothing changed so I can't post any oldPlac &/or newPlac names listed from your Records Window

[tatewise]

Sorry, my mistake in the report generating code.
Try the attached Rearrange Address and Place Parts plugin Version 1.6.3 Date 21 May 2021.

If any plugin reports an error like that and offers to Continue or Exit, etc.
Always choose Continue otherwise FH will Exit.
However, it is FH that will Continue and NOT the Plugin because it has fatally failed.

When we have resolved the current problem, I will explain how to move Address parts into Place parts.
Having reviewed the Help advice page I can see it is misleading and needs rewording.

[BakerJL75]

This will be more helpful. It wouldn't let me copy/paste so hopefully a screenshot will do:

Screenshot 2021-05-21 110550.png (36.37 KiB) Viewed 4683 times

Always choose Continue otherwise FH will Exit.

Thank you for the info. I had figured that out.

I will explain how to move Address parts into Place parts.
Having reviewed the Help advice page I can see it is misleading and needs rewording.

I usually have to read it a few times, but usually get there.

[BakerJL75]

Oops there was more.

2.png (16.83 KiB) Viewed 4682 times

3.png (18.68 KiB) Viewed 4682 times

And kept going like that. I will click through all and let you know if I get anything else.

[BakerJL75]

And the data you wanted. If CVS doesn't work for you let me know and I'll change it.

Rearrange Address and Place Parts Summary.csv

(119.45 KiB) Downloaded 124 times

[tatewise]

Thank you for that feedback Jackie. There is nothing obviously unusual in the Place names you provided.
The Error Report Data identifies with question marks (?) the missing data that causes the error.
However, the script that creates that data appears to be foolproof, so I still don't understand the root cause.

Could you please perform some more tests that may throw some light on the problem?
If you run the plugin with the same settings on the same Project data does it always produce the Error Report Data for the same Place name?
i.e. run the plugin on a copy of your Project, afterwards use Edit > Undo Plugin Updates, then run the plugin again, and so on.

If it is the same Place name every time, then please provide details of its usage Links.
i.e. Select that Place name in the Records Window, run the Where Used Record Links plugin, and post the Result Set here.

Alternatively, I could Email you so you could reply with your GEDCOM file attached. Would that be OK?

BTW:
I've realised that this Plugin needs enhancing to handle Rich Text Place record links and Source Template Place Metafields and it is possible that those omissions are behind the problem we are investigating.

[BakerJL75]

I exported a GEDCOM (FH Export) with no living people. I re-imported it and it does the same thing, just at a different place. I'd prefer to email it just in case someone living accidentally got included.

[tatewise]

I've investigated this with Jackie's help and it is a combination of the BEWARE MoveNextSpecial skips Citation Metafields (19447) problem plus use of new FH v7 features and some unusual Place names that only differ by one space.
The plugin now works Ok with cures for those issues but I won't publish until the MoveNextSpecial problem is fixed.

[BakerJL75]

Thanks Mike

[johnhanson]

Mike
I have just tried using the "rearrange address place parts" plugin on my master database

I had simply selected the "Shift all places left" and clicked the perform button

The original database is from TMG and has the address incorporated with the place hence the 29,000 place

mind there are 35,000 people in the database

I tried it on small subset first to make sure that it would do what I wanted - which it did

There were no error messages, no warning - it just disappeared from the screen appearing to crash

[tatewise]

John, please confirm you are running the latest plugin Version 1.6.
Earlier versions may not always run well with FH V7.

After clicking PERFORM REARRANGEMENT of Parts the plugin should start a progress bar with such a large number of Places.

Are you saying just the Plugin crashed or that FH crashed as well?

It is possible that the large number of Places is causing memory problems inside the Plugin. That has happened in other Plugins but I have not yet modified this Plugin to handle that possibility.

[johnhanson]

Mike
Running 1.6 - I would not have used ths thread otherwise

No progress bar appeared and FH just crashes

Happy to let you have copy of project if need be

[tatewise]

If necessary I may need your Project GEDCOM.

Please try the attached Rearrange Address and Place Parts plugin Version 1.6.4 Date 01 June 2021.

That fixes some minor issues and should handle the memory problem.

[johnhanson]

Mike
installed and ran 1.6.4

Still crashed FH

No pop-up screen but seemed to take longer to get there

as a waring the project as a whole is seems to have grown massively - is now nearly 400mb and there is no media

A lot seems to be snapshot files - not sure if I can just delete them

[tatewise]

If you are talking about .fhss files in the Snapshots sub-folder then they can be deleted.
FH should purge old ones (assuming you have default settings for Snapshots) but sometimes fails to do so.
FH will create a new Snapshot file for each day that FH is run and delete an old one.

If you have my direct Email address then please send your GEDCOM file (not the Project) as an attachment.
Otherwise, use the Contact option on the right of this posting to send me a message and we can take it from there.

[johnhanson]

Mike
Have sent you the Gedcom as a zip file - still 6mb but better than the 42mb Gedcom

Regards
John

[tatewise]

Thanks John.
I have your GEDCOM and can repeat the symptoms. So now I just need to discover why!

[johnhanson]

Mike
I doubt it is something as simple as running out of resources!

Nothing obvious shows up when looking at it in Excel

There are a couple of supplementary issues with the places but I will raise them as a separate issue after it is resolved as not related

Thanks for the effort in advance

No major hurry as I need to finish my sources lecture for the weekend and the PC goes in for an overhaul on Tuesday!

[tatewise]

John, I have a mixture of good news and bad news.

After several experiments, I think the Rearrange Address and Place Parts problem is one of exhausting memory, although I am disappointed that FH simply crashes without warning. If I enlarge the plugin script with diagnostics then it fails earlier. If I delete unused fragments of the plugin script then it fails later. No amount of memory management garbage collection fixes the problem. Perhaps the problem should be reported to Calico Pie, but I need your permission to send them your GEDCOM file.

The change you wanted is to delete the <blank> first comma separated part of every Place record.
That can be achieved with the Search and Replace plugin with the following settings.
Set the Search Scope to Place Records (_PLAC)
On the Extra Filters tab, untick everything except Record Names/Tiles and choose Place Name (_PLAC.TEXT)
Back on Major Options tab, enter ^, ? in the Seach: box and select LUA Pattern Mode
Untick Confirm the action for every item found and click Search & Replace
However, before you do that see below...

SearchAndReplacePlaces.png (56.94 KiB) Viewed 2623 times

When I first tried that, some Place names produced an error message and were not changed.
It turns out that you have many Place names that are duplicates, which need to be merged.
Those that only differ by a space after 1st comma become the same Place name when Search and Replace is run and that is not allowed so they fail.

I strongly advise you run the Find Duplicate Place Names plugin attached below and use Edit > Merge/Compare Records... or Tools > Work with Data > Places... and Merge to eliminate the duplicates before anything else.

[johnhanson]

Mike
Thanks for the efforts and the answer which is partly what I expected
Happy for you to send the file to Calico Pie - have known Simon since version1 - Yep I was one on those that actually saw it prior to initial release

Have used the plugin to sort out the duplicate entries - thank you.

However in think that some of the place name duplication is partly down to Calico Pie

There is no consistency in placing spaces after commas so sometimes you can get one and sometimes not

The classic on the list was Selby, Yorkshire and Selby , Yorkshire - I would expect the program to be consitsant in the storage. They don't appear of course in report, etc as the program does the tidy up first - Or "Rochester Place" where one of the entries had two spaces not one

If you have a place name as ,,, Selby, Yorkshire, England and edit it you can make it ,, , Selby Yorkshire, England and it just adds it as a new entry and doesn't complain that it allready exists.

Thanks again for you efforts

Will sort out removing the first field the way you mention after the PC comes back from its upgrade on Thursday

[tatewise]

I have raised Calico Pie ticket #254080 and confirmed that it fails the same way in FH V6.2.7.
Their reply gave some useful clues and after a little trial & error, I may have fixed the problem ~ let me finish testing.

IMO: The Place name duplication is more user error than down to FH.
FH treats every Place name as a simple text string where commas and spaces are just as important as any other character.
So any difference in the way a Place name is entered is considered a new Place that demands a new Place record.
It is much the same as any other records that can have similar names/titles.
FH does not warn that there is an Individual, Source, or Media record with a similar name/title when you add a record.

Theoretically, when you start entering a Place name, the FH auto-complete should suggest any matching similar Place.
Unfortunately, with a fixed comma part format, the number of leading commas and spaces must be the same as when the Place name was entered before, otherwise the auto-complete is ineffective.
That prompted the lengthy Find Duplicate Place Names (19048) topic discussion and my plugin.

It could be argued that the FH Place auto-complete should be more tolerant of commas & spaces but the rules are not obvious.