* Map Life Facts 3.3/4/5 questions

[tatewise]

The attached Plugin fixes the first problem above.
It was due to sometimes writing back an unchanged value of a Place record field.

The attached Plugin deals with the second problem by recreating what appears to be a missing Place entry.
But it should rectify that when it loads the Place Records from FH.

Could you also send your Map Life Fact.loc file and the Places part of the GEDCOM file.
Attach them to this thread if possible.

[BillH]

Mike,

I have attached the two files. I had to zip them as they were too large to upload as-is.

Bill

[BillH]

Mike,

Everything looked fine using [Address] ~ Place and Plugin Data Folder. I switched to Place Fields only and Place Records. All OK.

Then I erased the Address ~ Place database, used Copy Place Database and closed the plugin. When I tried to restart it, I got the same You are recommended to save changes to the current project... message window. I selected NO and the plugin started up.

I changed to [Address] ~ Place and everything again shows as No Data and the lat/long values are again missing from the .loc file.

Bill

[tatewise]

Thank you for your two files.
With them I am getting the same error scenario as you have been reporting for a few days.
So now I should be able to discover the cause.

If it is possible for you to experiment on a much smaller Project, then that would give a clearer picture of what is working and what is not.

I think the problem, whatever it is, is disrupting the normal behaviour of the Plugin.
Whether it is the quantity or the use of certain characters in Place names I have yet to determine.

[BillH]

Mike,

I'm glad you were able to reproduce at least some of the problems. At this point I think there are three outstanding issues... I'm not sure if you were able to reproduce all of them or not.

1. Clearing out the lat/long values in the .loc file thereby making everything No Data.
2. The error window for attempt to index field '?' (a nil value).
3. The window from FH saying to save the project before launching the plugin even though I haven't changed anything.

I'll try to create a smaller project tomorrow and see if I can get the same symptoms.

Bill

[tatewise]

3. I believe I have fixed.

1. & 2. I think are symptoms of the same problem.
I can now see how the problem arises.
It is a combination of a large database, a slight coding mistake that is amplified by the large database, that makes the Plugin run extra slowly.
In turn, that allows you to change tab before it has finished, and there should be code to prevent that.

If I am right, it should be easily fixed tomorrow.

[BillH]

Mike,

I just want to make sure. In your message of Thu Jan 01, 2015 3:37 pm you indicated that the one issue had been resolved with the version of the plugin you posted. However, I downloaded and installed that version and still got the problem as I reported in my message of Thu Jan 01, 2015 4:30 pm. I wasn't sure if you had fixed it again after my last report of the problem.

Bill

[tatewise]

The main problem, that with your help I have now isolated, allows two Plugin tabs to be active at the same time, namely Set Preference Options and Geocode Location Plots. The outcome of that is somewhat unpredictable, and I believe can lead to all three symptoms you mentioned on Fri Jan 02.

Separately, the third symptom was also caused by a coding error which I did fix.

Until I fix it, the workaround to the main problem is when you switch Preference modes, wait until all Status: messages have finished and it says OK and all the buttons nearby are not greyed-out but their normal green colour. Then it is safe to change tab to Geocode Location Plots.

[BillH]

Mike,

Glad you were able to figure out what was happening. I was waiting till the status said OK, but wasn't paying attention to the buttons. I'll make sure I wait until they have returned to their green color.

Bill

[BillH]

Mike,

OK... now I feel like I'm going a bit crazy.

I deleted the .loc file. I started the plugin with it set to Place Records. I went to the Database Management tab and clicked on Copy Place Database. Shouldn't this have put my places into the [BOTH] table? I closed the plugin and opened the .loc file and all three tables were empty.

Bill

[tatewise]

Well spotted Bill. Another wrinkle associated with using Place Records just for Place Fields only and Plugin Data Folder for everything else!

I hope the attached Plugin V3.5 dated 02 Jan 2015 fixes most if not all the problems.

The irony is that I was contemplating inhibitting the busy tabs, and investigating why so often the database needed saving, when the unusual errors started arising, and that threw me off the scent. But I think we have got there in the end.

[BillH]

Mike,

Looks good so far. I've been playing around for about 15 minutes and don't see any of the symptoms that I was seeing before. I ran through all the steps including deleting the .loc file and starting all over again.

I do have a question though. I had the options set to Place Fields only and Place Records. I changed the Locations From: to [Address] ~ Place. The plugin never gave me a chance to select anything for Database In: It quickly changed from Place Records to Source & Repository and then to Plugin Data Folder and seemed to be processing and then came back with Status: OK. Since I would only want to use Plugin Data Folder when using any of the Address ~ Place options, this works for me. But if someone wanted to use one of the other Database In: values, what would they do?

Well done on getting the problems corrected.

Thanks,
Bill

[tatewise]

I am relieved that it is all going well so far.

The Database In: display sequence you describe is a side effect of multi-thread processing, and you may be interested in the details:
Because [Address] ~ Place does not support Place Records, that option must be removed from the Database In: droplist, so the Lua script tells the IUP GUI to remove it.
Now there are two threads running in a race: Lua getting on with its script, and IUP changing the droplist.
If IUP wins, it can no longer display Database In: Place Records so displays the next available up the droplist, which is Source & Repository.
Then the Lua script gets a go, and tells IUP to set Database In: to Program Data Folder.
Eventually, when IUP has another go, it updates the display.
If the PC happens to be busy doing other tasks at the same time, then it will be even more likely.

I had notice the effect sometimes myself and made a mental note to look at it.
Now I have looked at it, and realise I can fix it.
When the Lua script needs to remove the Place Records option, it knows what the required option needs to be, so it can set that Database In: choice before removing Place Records.
That will be in the Plugin Store release next week, if your testing does not find any further issues.

To answer you last question. If a user wants one of the other Database In: options, then they just select it and that then becomes the default storage mode, except when Place Fields are synchronised to Place Records.

[BillH]

Mike,

Thanks for the explanation. Very interesting.

I've been playing around off and on today and I am not able to make it break.

I'll keep using it and let you know if I see anything, but it looks like it is good to go.

Thanks again,
Bill

[BillH]

Mike,

I've been playing around a bit more and came across several things I can't explain. Can you explain why these would happen like this?

First Scenario -

1. I had the options set to [Address] ~ Place and Plugin Data Folder. I erased the Address ~ Place database and then copied it again using Copy Place Database. The stats showed 82 Plotted, 2198 Defined, 951 No Data, 3231 Total.
2. I switched to Place Fields only.
3. I switched to Address ~ Place. The stats showed 0 Plotted, 0 Defined, 951 No Data, 951 Total.
4. I switched to [Address} ~ Place. The stats had changed even though I hadn't geocoded anything. Now the stats showed 82 Plotted, 2189 Defined, 960 No Data, 3231 Total.

Why did the Defined drop from 2198 to 2189 and the No Data go up from 951 to 960? This is repeatable using my database as it now stands which is a bit different than the files I posted.

Second Scenario -

1. I had the options set to [Address] ~ Place and Plugin Data Folder. The stats showed 82 Plotted, 2198 Defined, 951 No Data, 3231 Total.
2. I switched to Address ~ Place. The stats showed 0 Plotted, 0 Defined, 951 No Data, 951 Total.
3. I geocoded the No Data locations. The stats showed 950 Plotted, 0 Defined, 1 No Plot, 0 No Data, 951 Total.
4. I switched to [Address] ~ Place. The stats showed 1032 Plotted, 2189 Defined, 1 No Plot, 9 No Data, 3231 Total.

In Step 4, why did the No Data total increase from 0 to 9 just by switching locations.? I'm not sure if this is repeatable, I exceeded the daily quota for geocoding.

Thanks,
Bill

[tatewise]

I am not entirely sure why that is happening, although I suspect it may be to do with Place Fields only with Place Records retaining all characters unchanged and listing Records that have no associated Place fields.
Whereas all other modes collapse multiple spaces single characters and only list actual Place and Address fields.

In both scenarios 9 locations are migrating from Defined to No Data.
Have a look at the 9 No Data in second scenario, both in the Plugin and in your Place Records, and let me know what you find.
Are there subtle differences in spaces, or something else unusual?

One thing to do though when selecting the Geocode Location Plots tab is wait a while and see if the Statistics alter after a few seconds. It may be another case of needing to inhibit switching tabs while the Plugin is busy assessing locations.

I have spotted one anomaly while looking at this. If you edit the Place name in a Place Record, then the Place Fields only with Place Records mode retains both variants of the name. Currently that only gets fixed by switching to Plugin Data Folder and back to Place Records.

[BillH]

Mike,

I am not entirely sure why that is happening, although I suspect it may be to do with Place Fields only with Place Records retaining all characters unchanged and listing Records that have no associated Place fields.

I wasn't sure what you meant here when you said records that have no associated place fields. Are there place records that have no place fields?

I don't see anything unusual about the 9 places that became No Data. Here are the 9 places that changed:

, Bremen,, Germany
, Malangen, Malangen, Tromsø, Norway
,, Wales
Ballymena, County Antrim, Ireland
Glasgow,, Scotland
Grønstad, Etne,, Hordaland, Norway
North America
Southampton,,, England
Weymouth,, Dorsetshire, England

One thing that was confusing is that I thought everything submitted to the geocoder became Plotted, Defined, No Plot or Invalid. I didn't think they could become No Data.

Another confusion was that I only told it to gecoode anything in the second scenario, not the first one. Also, I only told it to geocode the Address ~ Place combinations and none of these that became No Data had addresses.

I have been waiting and the only place I really see the numbers take a while to update on the Geocode Location Plots tab is after using the erase and copy buttons on the Database Management tab.

Bill

[tatewise]

You asked: "Are there place records that have no place fields?"
Yes, you can create Place Records without linked Place fields anywhere else in your data.
Just like you can create Source Records wihout any linked Citations.
In other words without any =LinksTo().

Yes, the only time you will see the Statistics change by themselves is when you switch from Set Preferences Options tab to Geocode Location Plots tab. Either after switching to/from Place Records or after using the Database Management buttons. The Plugin has to compare all the appropriate locations in FH with those in the Plugin. It may add to the Plugin any it finds in Place Records, and Place &/or Address fields, or remove any that do not exist in Place &/or Address fields.

So I suspect what is happening is that there are 9 Place Record with Defined plots that when copied to Address ~ Place initially appear as Defined, but then get removed and replaced by similar but not identical Address ~ Place locations that therefore have No Data. As I said, I think it may be due to subtle differences in Place names, so review those 9 carefully in BOTH the Place Fields only mode and the Address ~ Place mode. Alternatively, it may be due to the Address & Place combinations we have investigated before. What if those 9 Places never exist on there own, but always with an Address. Then there will be no ~ Place entry to match each Place Fields only entry.

[BillH]

Mike,

Thanks for the explanation on place records with no place fields. I knew you could have place records that weren't link anywhere. I just hadn't thought of calling these place records with no place fields. None of the mysterious 9 places is in this category. They all have at least one linked event. Interestingly they are all used in either immigration from or migration to fields. However, I have many of both of these that are not changing mysteriously.

Yes, the only time you will see the Statistics change by themselves is when you switch from Set Preferences Options tab to Geocode Location Plots tab. Either after switching to/from Place Records or after using the Database Management buttons. The Plugin has to compare all the appropriate locations in FH with those in the Plugin. It may add to the Plugin any it finds in Place Records, and Place &/or Address fields, or remove any that do not exist in Place &/or Address fields.

Could this happen even though nothing had changed in FH in the meantime? All I did was switch locations and they changed. I hadn't changed anything in FH.

So I suspect what is happening is that there are 9 Place Record with Defined plots that when copied to Address ~ Place initially appear as Defined, but then get removed and replaced by similar but not identical Address ~ Place locations that therefore have No Data. As I said, I think it may be due to subtle differences in Place names, so review those 9 carefully in BOTH the Place Fields only mode and the Address ~ Place mode. Alternatively, it may be due to the Address & Place combinations we have investigated before. What if those 9 Places never exist on there own, but always with an Address. Then there will be no ~ Place entry to match each Place Fields only entry.

I have looked at these over and over and don't see anything strange. As far as I can see they look the same in Place Fields only mode and Address ~ Place mode. Here is what one of them looks like in the BOTH table and in the PLAC table.

[" ~ Grønstad, Etne,, Hordaland, Norway"]={1293},
["Grønstad, Etne,, Hordaland, Norway"]={4816},

None of the mysterious 9 are ever referenced with an associated address in FH.

Bill

[tatewise]

So just to confirm, when you switch from Set Preferences Options tab to Geocode Location Plots tab, those 9 start as Defined and automatically 'migrate' to No Data?

And 'YES', nothing needs to have changed in FH if the Plugin thinks the 'migration' is needed.

To complete the picture, what is in table {1293} and table {4816} in the .loc file?

If you can give a few more of the 9 with their tables that might give me some clues.
Is it possible that there are other similar locations whose names only differ by an odd space, so that there are two virtually identical Place names in the Place Records, and one is Defined and the other is No Data?

On re-examination, it is not multiple spaces as the Plugin handles that OK.

[BillH]

Yes, that is correct. They switch from Defined to No Data.

OK... today is very weird. If I leave the options on [Address] ~ Place and Plugin Data folder and I repeatedly use Erase Addr ~ Place Database and then Copy Place Database, I sometimes get 2198 Defined and 951 No Data and sometimes I get 2197 Defined and 952 No Data. The location that sometimes ends up in Defined and sometimes in No Data is:

, Malangen, Malangen, Trømso, Norway.

This does not change every time I do it, but it changes back and forth fairly frequently.

Here are all 9 places before I switch around the database and they get mysteriously moved to No Data. After they get moved of course the BOTH table values are all blank.

[" ~ , Bremen,, Germany"]={2035},
[", Bremen,, Germany"]={5496},
-- Table: {2035}
{
["sub"]="",
["lng"]="8.8333333",
["lat"]="53.0833333",
["mod"]="Defined",
},
-- Table: {5496}
{
["sub"]="",
["lng"]="8.8333333",
["plc"]=1101,
["lat"]="53.0833333",
["mod"]="Defined",
},


[" ~ , Malangen, Malangen, Tromsø, Norway"]={1386},
[", Malangen, Malangen, Tromsø, Norway"]={3260},
-- Table: {1386}
{
["sub"]="",
["lng"]="18.6",
["lat"]="69.4",
["mod"]="Defined",
},
-- Table: {3260}
{
["sub"]="",
["lng"]="18.6",
["plc"]=1701,
["lat"]="69.4",
["mod"]="Defined",
},


[" ~ ,, Wales"]={1265},
[",, Wales"]={4383},
-- Table: {1265}
{
["sub"]="",
["lng"]="-3.7526304",
["lat"]="52.3808901",
["mod"]="Defined",
},
-- Table: {4383}
{
["sub"]="",
["lng"]="-3.7526304",
["plc"]=2273,
["lat"]="52.3808901",
["mod"]="Defined",
},


[" ~ Ballymena, County Antrim, Ireland"]={2174},
["Ballymena, County Antrim, Ireland"]={3959},
-- Table: {2174}
{
["sub"]="",
["lng"]="-6.276389",
["lat"]="54.8715593",
["mod"]="Defined",
},
-- Table: {3959}
{
["sub"]="",
["lng"]="-6.276389",
["plc"]=1169,
["lat"]="54.8715593",
["mod"]="Defined",
},


[" ~ Glasgow,, Scotland"]={1688},
["Glasgow,, Scotland"]={4914},
-- Table: {1688}
{
["sub"]="",
["lng"]="-4.2484786",
["lat"]="55.8618449",
["mod"]="Defined",
},
-- Table: {4914}
{
["sub"]="",
["lng"]="-4.2484786",
["plc"]=1995,
["lat"]="55.8618449",
["mod"]="Defined",
},


[" ~ Grønstad, Etne,, Hordaland, Norway"]={1318},
["Grønstad, Etne,, Hordaland, Norway"]={4796},
-- Table: {1318}
{
["sub"]="",
["lng"]="6.0080836",
["lat"]="59.68589",
["mod"]="Defined",
},
-- Table: {4796}
{
["sub"]="",
["lng"]="6.0080836",
["plc"]=2187,
["lat"]="59.68589",
["mod"]="Defined",
},


[" ~ North America"]={696},
["North America"]={4358},
-- Table: {696}
{
["sub"]="",
["lng"]="-102.8793629",
["lat"]="58.7157977",
["mod"]="Defined",
},
-- Table: {4358}
{
["sub"]="",
["lng"]="-102.8793629",
["plc"]=2185,
["lat"]="58.7157977",
["mod"]="Defined",
},


[" ~ Southampton,,, England"]={732},
["Southampton,,, England"]={3909},
-- Table: {732}
{
["sub"]="",
["lng"]="-1.4044295",
["lat"]="50.902644",
["mod"]="Defined",
},
-- Table: {3909}
{
["sub"]="",
["lng"]="-1.4044295",
["plc"]=1325,
["lat"]="50.902644",
["mod"]="Defined",
},


[" ~ Weymouth,, Dorsetshire, England"]={24},
["Weymouth,, Dorsetshire, England"]={5470},
-- Table: {24}
{
["sub"]="",
["lng"]="-2.4569143",
["lat"]="50.6194895",
["mod"]="Defined",
},
-- Table: {5470}
{
["sub"]="",
["lng"]="-2.4569143",
["plc"]=1277,
["lat"]="50.6194895",
["mod"]="Defined",
},

Thanks,
Bill

[tatewise]

Referring back to my previous posting (which I have been editing) can you check there are no duplicate Place Record entries for these 9 Places that only differ by the odd space, so that when despaced they would be the identical.

Do some have plotted Lat/Long and others no Lat/Long?

[BillH]

Mike,

No, there are not places with similar names as you describe.

Bill

[tatewise]

I am at a loss to explain your symptoms.
I know you said there were no Addresses, and the .loc file supports that, but can you double check the Address field paired with each Place field, linked to those 9 Place Records.
Does it exist at all?
If so, what is its contents?
Check the GEDCOM if necessary to see if some odd control chars or something are present.

Attached is another Plugin version dated 04 Jan 2015.
It fixes the busy tab problem, and the Place Record name change problem.
It also reports in red on the Status: line any location names that are set to No Data or are Deleted.
It pauses on each one, so if there are a lot it may be quite slow, but may give an insight into what is happening.

[BillH]

Mike,

Boy... I've looked and looked and still see nothing odd or unusual. In the GEDCOM file they look fine. No control characters or anything.

In FH I looked at every link and there are no address fields associated with any of these 9 places.

I'll download the new version and see what happens.

Thanks,
Bill