* Find Duplicate Individuals - window taking focus

[BillH]

Mike,

Somewhere along the line the progress window has again started to take focus so you can't do anything else while the plugin is working. I thought this had been broken and fixed in an earlier version.

Thanks!

Bill

ID:6879

[tatewise]

Bill, I have not changed this Plugin for at least 3 months, but will review that symptom at the next update.

[BillH]

OK... thanks Mike.

Bill

[BillH]

Mike,

Just wondered if you have had a chance to look at this.

Thanks,

Bill

[tatewise]

Hi Bill,
[strike]I have created a candidate Version 3.2 Plugin in my SkyDrive at Find Duplicate Individuals.[/strike] [Edited]
It should overwrite your Version 3.1 downloaded from the Plugin Store.

This should fix the Progress Bar taking focus problem.
It adds Cremation as an alternative to missing Death or Burial events.
It has a slight evolution of the Version 3.1g method of managing its database.
It loads the data for all Individuals in the same Pool as soon as it encounters the first member of the Pool.
e.g.
The first Individual has RecId=1, and all the relations in the same Pool will have their data loaded.
Subsequent Individuals will not need their data loading until a member of a different Pool is reached.
The scoring process between pairs of Individual data has not changed.

Let me know how it performs, and if all OK, I will submit it to the Plugin Store next week.

Remember you can revert to Version 3.1 from the Plugin Store, and still run WiP Version 3.1g alongside.

[BillH]

Mike,

I installed version 3.2.  I cleared the soundex cache and then ran the plugin and got the following error:



A question also.  Once you use the Set the Updated from Date to this last run Date, is there anyway to revert it to not have a date?

Thanks,

Bill

[tatewise]

Sorry about that 'Error Message' Bill, it appears to be caused by a combination of things.
Firstly, the new data loading techniques introduced about Version 3.1f and extended in Version 3.2.
Secondly, a relationship loop in your data, perhaps where a Descendant is their own Ancestor, or maybe where a pair of siblings have married another pair of siblings.
e.g. a Child is their own Grandparent, or brother A & B have married sisters C & D.
I am working on a fix.

You can reset the from Date by selecting the part you want to change and entering any valid Date value.
Alternatively, use Set Preferences tab and click Restore GUI Defaults, but this also resets the Include any Selected Subset of the Individuals.

[BillH]

Mike,

I have been using version 3.1 from the store so I wasn't running into any problems.

I do have several cases where siblings married siblings and probably have the other condition as well.

I don't seem to be able to update the Set the Updated from Date to this last run Date field. I can set the Include Individuals last Updated from this Date field. Maybe I don't understand what the Set the Updated from Date to this last run Date field is for. I wanted to go back and check everyone in my database, not just those with recent updated dates. I read the help, but really couldn't figure out the difference between these two fields.

Bill

[tatewise]

The Set the Updated from Date to this last run Date button sets the Include Individuals last Updated from this Date field in order to exclude already checked Individuals.
To include everyone, use the Set Preferences tab, and click Restore GUI Defaults.
Or edit the Date as you have been doing.

My 'Error Message' problem analysis was slightly wrong, and discovered it is caused by single parent Family Records that have not been Saved.
After the data has been Saved and re-Opened the problem appears to vanish, but I have added a check to avoid the problem scenario.

[strike]A new candidate Version 3.2 Plugin is in my SkyDrive at Find Duplicate Individuals.[/strike] [Edited]

[BillH]

Hi Mike,

Strange... I had done a save before running the plugin and still got the error. In any case, I downloaded the new version and it worked so the problem is corrected. Also, the progress window isn't taking focus.

Also, thanks for the explanation on the dates.

Bill

[tatewise]

You have to do more than just a Save.
You must Save the Project, then close FH or switch to a different Project, and then re-Open the original Project.
The Project reload is needed to clear the empty link to the missing Family parent, as per normal FH practice in clearing blank fields.

You can still see the problem symptoms by running the WiP find_duplicate_individuals Version 3.1g with a new single parent Family after performing just a Save.

[johnmorrisoniom]

I have just downloaded v 3.2 from the store, but when I try to run, it gets stuck at 0% on loading soundex cache.

[tatewise]

John, the way Plugin now works means that for large databases it may take some time to progress past the first Individual.
This is because it loads data for all family relationships of that first Individual before proceeding, but hopefully this saves time later.

How long have you left it running?

Can you try it on a smaller database?

If my analysis above is correct, then I may need to review the Progress Messages to cope with large family pools.

[tatewise]

I have revised the Progress Bar so that it advances while initialising the data, and the Record Id message is followed by dots indicating loading activity.

[strike]A new candidate Version 3.3 Plugin is in my SkyDrive at Find Duplicate Individuals.[/strike] [Edited]

[johnmorrisoniom]

Hi Mike.
Vers 3.3 took about 20 mins with no obvious progress apart from the advancing dots above the progress bar, then the progress bar kicked in. The whole run took about 55 minutes to check the 1486 records added since 8th may against the total of 37000+. It did find a couple of definite matches, one was a family with 8 children , but with surname spelt with and without an e at the end. The mother only scored 35 though despite the number of children being the same, husbands name the same birth dates were only 1 year apart and death and burial dates were exact matches.

[tatewise]

I should be able to make the Progress Bar advance better during the initial dotty phase - I will have a look.

I am a little surprised the run took so long.
Did you set the Include Individuals last Updated from this Date to about 1 May 2013?
If not, then all 37000+ would have been assessed against each other, which matches the 55 mins run time.

The explanation for the mother's score of 35 goes a bit like this:
Assuming her first name matches perfectly (6 points) and surname Soundex matches (2 points) and no other names match then total is 6 points.
If her husband and best matching child have similar name matches, they will score 6 points each.
You do not mention parents, so assume they score nothing.
Birth dates a year apart would score nothing.
Death/Burial dates perfect match would score 6 points and if place names match could reach 10 points.
You don't say how many of the family had matching dates.
The current assessment only takes into account the best matching child, regardless of how many children.
So the total is 3 x 6 for names + 10 for a death date & place = 28 points, which is not far short of 35.

If you had used Show the previous Result Set of Duplicates in FH with Enable Diagnostic Mode ticked then a breakdown of scores would been shown.
But if you have now Merged the Duplicates those details will be hidden.

[tatewise]

[strike]A revised candidate Version 3.3 Plugin with Last Updated: 09 June 2013 is in my SkyDrive at Find Duplicate Individuals.[/strike] [Edited]

This version does away with the dotty ... progress indication, because I felt it was using too much run-time.
It now simply says Loading Relation Pool please wait, because that is what it is actually doing.
Also the Progress Bar should move forward more than before, as well as the time progressing.

[BillH]

Mike,

I really like the way you are indicating progress now rather than the dots. Works great.

Thanks!

Bill

[tatewise]

Thanks Bill. I presume its run-time is similar to earlier versions.

Once John Morrison has given it the OK, I will publish it in the Plugin Store.

BTW: Have you tried my candidate Lookup Missing Census Facts Version 2.7 available via Lookup Missing Census Facts ~ New Plugin?

[johnmorrisoniom]

Hi Mike,
I've not had a chance to run the latest version yet (maybe I'll get myself caught up when they come up with a 48hr day)but should be able to check it out tonight.

[johnmorrisoniom]

Hi Mike,
Latest version took 1387 secs (Just under 24 mins) to check 319 individuals against a total file of 37,167. I forgot to take a screenshot of the plugin menu before I ran it, so this is next run just after it produced the result set.



so the top date was originally 5th June

[tatewise]

I guess at the beginning it still took about 20 minutes Loading Relation Pool please wait.

Despite being a bit slow, are you happy with the way the Plugin performs?

Could you also check out the Lookup Missing Census Facts candidate Plugin, which I believe offers the Birth Year feature you requested last month for Before/After Dates, etc.
In fact it does a lot more...

[johnmorrisoniom]

Hi Mike,
Yes both plugins work very well.
The lookup missing census facts I use almost every day. Duplicate Individuals I use about once a month

[goodwin2]

Hi,

I just tried to run this plugin. Knowing that I have a large database, I let it run - and run - and run - somewhere in the neighborhood of 1.3 hours. And then it hung my computer. Should I just assume that it looks at so much that it won't work with a database of over 40,000?

Still working at using ALL the good stuff with FH5.

[johnmorrisoniom]

Hi,

Should I just assume that it looks at so much that it won't work with a database of over 40,000?

Still working at using ALL the good stuff with FH5.

My database is well over 40,000 and I don't have any major problems, I do find though that there is quite a difference between similar computers with different OS. Win 7 64 bit machine is significantly faster run time than my XP machine.