Map Life Facts V2.0

Post by **tatewise** » 16 Jan 2012 10:40

Version 2.0 is now available from the Plugin Store, and includes many enhancements:
1. Redesigned GUI with embedded Google Map, option tabs, progress bars, tool tips, etc, etc.
2. Updated online Help & Advice from the FHUG Knowledge Base.
3. Uses the Google Geocoder API that is about 10 times faster.
4. FH Beta V5.0.0.3/4 code fixes and revisions.
5. Tested on Windows 7, Vista, & XP.
6. Fixed all known problems such as 'freezing', 'Firewalls', etc, etc.

Thanks to Jane for all the help and advice.

ID:5810

YouTube · Post by **Jane** » 16 Jan 2012 15:57

Mike, just trying your Map Life Facts and I seem to have a problem as it starts and then uses one of my processors flat out for about 30-50 seconds and then starts up, but the buttons don't fit on the screen and I can't seem to resize. Do I need to delete a settings file?

BillH · Post by **BillH** » 16 Jan 2012 17:47

Mike,

I just tried the new version and had the same results. Locked up one processor for over a minute and then the window wasn't big enough to show everything.

Bill

johnmorrisoniom · Post by **johnmorrisoniom** » 16 Jan 2012 18:46

Hi Mike,
Ran OK on My Laptop (Acer Aspire 7741 with i3-380M, W7 home premium x64, 4GB ram, screen resolution 1600x900) all buttons visible and no lockups, but tested on a small (27 entries) file.
Mapping places , all seemed OK.
Mapping addresses, not so good, as over half were deemed to be in the USA, and not the UK.
I will test it on my desktop tomorrow.

Post by **tatewise** » 16 Jan 2012 21:37

Sorry about those problems.

In the past, I had something similar to Jane and Bill regarding button size/window width, but thought I had fixed it.
I believe it is related to long Place/Address fields in the Location drop-down list.
I am working on a Plugin version that allows the window to be resized, which may provide the solution.

Regarding the long & busy startup - how many Individuals do you have in your GEDCOM data?
I think the slow startup may be caused by thousands rather than hundreds of Individuals.

John, I have little control over where the Google Maps Geocoder decides your Addresses are located.
Especially if your Addresses do NOT include England or UK.
However, it should give preference to the region where your PC is located.
I am thinking about ways to improve this, either by utilising the Place field along with the Address, or by telling Google Maps Geocoder to prefer locations in a particular region determined by a Settings Option.

johnmorrisoniom · Post by **johnmorrisoniom** » 16 Jan 2012 22:04

Hi Mike,
I would think that utilising Places and addresses together would be the better solution, as I, as well as many others have people from UK, USA and Australia, South Africa and New Zealand in their files.

YouTube · Post by **Jane** » 16 Jan 2012 22:42

Regarding the long & busy startup - how many Individuals do you have in your GEDCOM data?
I think the slow startup may be caused by thousands rather than hundreds of Individuals.

My file is pretty small compared to some I have seen, at just 3,500 people. You might consider not building on start up the geocoding section and loading the place list on demand, rather than starting with it, that way the process only needs to run when someone wants to do it, rather than always?

Just so if you want to do a quick map and you have already got all the places coded you don't need to wait?

BillH · Post by **BillH** » 16 Jan 2012 23:09

Mike,

I have 6788 individuals.

I am running Windows 7 Home Premium 64-bit on a i7-2600 with 8 GB RAM. My display is 1920x1080 on a 23' monitor.

Didn't have this problem at all starting up the previous version of the plugin.

Bill

johnmorrisoniom · Post by **johnmorrisoniom** » 17 Jan 2012 09:24

Hi Mike,
Tried the plug in on my Big File (26,000 individuals)on my desktop computer
W7 pro x64 4gb ram 8800sli graphics 2.4g quad core.

No problems with window size and buttons, but I had to crash FH after 10 mins as nothing appeared to have happened.
100% useage of one Cpu core.

version 1.1 was ok with this file

No problems at all with the smaller projects though

YouTube · Post by **Jane** » 17 Jan 2012 09:55

Mike, I notice you are using MoveNextSpecial, might be worth as you know all places and addresses will be down from an event doing a double loop rather than using MoveNextSpecial

Post by **tatewise** » 17 Jan 2012 14:47

That is not what is taking the time.
For some strange reason that I cannot resolve, it is the APPENDITEM attribute of the Location or Individual dropdown lists that's taking about 10 millisecs for each item!!!
i.e. about 1000 items per second - which is very slow!

The GUI definition is almost identical to your Map Geo Code Maintenance dropdown for Places.
If I change your PLAC to ADDR it populates the dropdown with over 2000 entries in a fraction of that time.

The other problems with the GUI I have sorted out, and added resizing of the window OK.

Post by **NickWalker** » 17 Jan 2012 16:14

Just as a warning of what you might face once the public get their hands on this.... I remember in the early days of my old Gedcom Census program I got a report of an error (which I fixed) which was because a user had something like 50,000 place names in their file! I've had users send me their GEDCOMs to illustrate errors with well over 50,000 individuals and also at one point someone with more than 12 wives recorded for one of their ancestors.

Post by **tatewise** » 17 Jan 2012 21:21

Thanks for that warning Nick.

Would you believe the reason why APPENDITEM is so slow is because the dropdown list has a tool TIP defined!?
I gave the Map Geo Code Maintenance dropdown Places list a TIP, and it runs slowly too.

To confirm I am on the right track, could you perform the following Map Life Facts V2.0 Plugin edits.

1) Locate the lstLocation iup.list dropdown definition, and replace its tip='Select...', parameter with visiblecolumns='5',
( Be careful to avoid the one that is in --[ [ comment ] ] and starts expand='YES', fgcolor=StrGreen, )

2) Locate the lstIndiName iup.list dropdown definition, and replace its tip='Individual...', parameter with visiblecolumns='5',
( Be careful to avoid the one that is in --[ [ comment ] ] and starts expand='YES', fgcolor=StrGreen, )

3) Locate the local function doBuildLocationsList() and after lstLocation.appenditem = strLocName insert iup.Flush()

4) Locate the local function doBuildIndiNamesList() and after lstIndiName.appenditem = strIndiName insert iup.Flush()

This should fix the buttons off screen problem, and load the Place/Address Locations and the Individuals dropdown lists at the rate of about 2000 per second.

BillH · Post by **BillH** » 17 Jan 2012 22:55

Mike,

I tried that and seemed to be partially successful. The window loaded and I was able to see all the of items in the window.

However, I got the following error when I ran the plugin.

...y Historian 5.0 BetaPluginsMap Life Facts-2.fh_lua

attempt to call field 'flush' (a nil value)
stack traceback:
...y Historian 5.0 BetaPluginsMap Life Facts-2.fh_lua

in function 'doBuildLocationsList'
...y Historian 5.0 BetaPluginsMap Life Facts-2.fh_lua

in function
(tail call): ?
[C]: in function '?'
...y Historian 5.0 BetaPluginsMap Life Facts-2.fh_lua

in function 'GUI_MainDialogue'
...y Historian 5.0 BetaPluginsMap Life Facts-2.fh_lua

in main chunk

Here is how I added the iup.flush(). Is that not correct?

for strLocName, tblLocData in pairs(TblLocationData) do
if tblLocations[strLocName] == nil then
if IntStorage ~= IntStoreFile then DeleteLocationRecord(tblLocData['rec']) end
TblLocationData[strLocName] = nil -- tblLocData = nil does not always work
else
lstLocation.appenditem = strLocName -- Populate the Location dropdown list
iup.flush()
end

Bill

Post by **tatewise** » 18 Jan 2012 00:04

Yes, that is correct except that flush should be Flush with a capital F.

My mistake in original posting, that I have now edited.

BillH · Post by **BillH** » 18 Jan 2012 18:28

Mike,

Made the change and now the plugin starts up fine in just seconds, doesn't seem to tie up a processor at all, and everything fits in the window just fine. Looks like this took care of the problem.

Are we supposed to be able to resize the main plugin window? Mine can't be resized.

Bill

Post by **tatewise** » 18 Jan 2012 21:27

Good to hear that works OK.
Presumably, that is with your 6788 individuals database?

The only window you can resize in V2.0 is the Help & Advice window.
V2.1 may well allow the main window to be resized.

Bill, could you check out V2.0 to see if it has cured the Comodo Firewall problems on your old PC.

BillH · Post by **BillH** » 18 Jan 2012 21:44

Yes, that is with my 6789 individuals file. The file has 1262 places.

It takes about 23 seconds from the time I click on Run till the buttons at the bottom of the plugin window become active.

In plotting the places I noticed two reasons why plotting was unsuccessful. One is 'Not Found'. The other is 'Invalid Request'. In your testing have you found any pattern as to when each of these types of errors occurs?

I was just wondering if either of these could be related to having extra commas to keep track of unknown parts of the place (parish, county, etc).

I will test this on my old Vista machine with Comodo and let you know what happens. I'll post the results to the other forum post.

Bill

Post by **tatewise** » 19 Jan 2012 00:00

I hope to improve the startup process in V2.1 for larger databases in various ways:
1) There will be progress bars to indicate what process is going on.
2) Some processes may be relegated to the background, or be stopped by the user, in order to do a quick map or two.

I need to improve the Help & Advice to explain the Not Found and Invalid Request status messages that come from the Google Geocoder.

Not Found means Google cannot find a matching location.

Invalid Request means Google does not like some non-alpha-numeric symbols in your GEDCOM Fact Place (or Address) entry.

In either case you have several options:
a) Change your GEDCOM data Fact Place (or Address) entry.
b) Enter a Substitute location in Plugin.
c) Enter known Latitude/Longitude values in Plugin.

BillH · Post by **BillH** » 19 Jan 2012 00:04

I did notice that most of the 'Invalid Requests' where for Norwegian place names with Norwegian characters. I'll have to manually set them up.

Thanks for all your help! Great plugin.

Bill

Post by **tatewise** » 19 Jan 2012 09:11

I have learnt a bit more about the Invalid Request and non-alpha-numeric symbols.
For technical reasons most non-alpha-numeric symbols have to be specially encoded in the Google Geocode URL.

I thought such symbols were being encoded correctly, but it appears that UTF-8 symbols with a code larger than 127 are incorrect and that is what Google is reporting.

I think I can fix the Plugin, so you should NOT need to change all your Norwegian characters.
Just wait for V2.1.

BillH · Post by **BillH** » 19 Jan 2012 17:28

Mike,

That is good news... I have a lot of these. I'll wait around to see if you are able to get the plugin to handle them.

Thanks,

Bill

Post by **tatewise** » 21 Jan 2012 22:35

I have discovered that the slowness of populating the Locations and Individuals dropdown iup.list controls, for large GEDCOM databases, does not apply to iup.tree controls.
The iup.tree controls also offer a better way of selecting items, similar to selecting files from folders.
Therefore, I am working on switching to these iup.tree controls and eliminating the startup delays.

BillH · Post by **BillH** » 25 Jan 2012 22:17

Hi Mike,

Just wanted to let you know that the new version 2.1 did fix the problem with the invalid requests. I went from 211 failed to 169 failed. Now the ones that are failing are because Google Maps can't find the place name as I've got it recorded. I tested by going to Google Maps in my browser and entering the same place name. I'll have to fix these up manually.

Thanks,

Bill

Family Historian User Group