* creating a file for Ancestry with no facts for living individuals

[tatewise]

Yes, I have been doing some testing myself, and that area of the Plugin needs revisiting.
It certainly does not suitably handle Date Ranges such as before or after dates.
I resolved similar problems with my Lookup Missing BMD/Census ... plugins, so will have to apply similar strategies here.

Remember that here it is only listing Forced Living folk without a Living Flag but who match the criteria.

Adding a Help page is something I was hoping to avoid, but may be necessary.

[BillH]

Mike,

I am a bit confused. I thought if I checked the Force Living Flag, it ignored what I had for living flag and used its own algorithm. Are you saying it will use the living flag if set and then for folks without a living flag it will use its own algorithm?

In any case, examples 1, 2, and 4 above did not have a living flag set. Example 3 did.

Thanks,

[tatewise]

The Living Flag is used unconditionally, and is why at the top it says Living People (Living Flag Set) Options.
Optionally, it will also Force Living Flag onto others who meet the criteria.
So the primary rule is "If Living Flag Set or if Forced Living Flag set then clean person".

[BillH]

Mike,

Thanks for the clarification on the the Living Flag being unconditionally used. I guess this helps to show that the plugin really needs a detailed help. For many of these options the option "name" itself doesn't give enough info on what is actually going on.

Thanks

[tatewise]

Try the attached Clean Living Persons Plugin Version 1.2.7 Date 27 Feb 2017.

The key changes are as follows :-

• The options dialogue has been restyled & reworded to make the rules easier to follow (I hope).
• The options follow a more logical order: 1) Privacy Rules, 2) Living Rules, 3) Cleaning Rules, and 4) Unused Records.
• The options now share their 'sticky' settings with the Clean Living People Plugin.
• Each drop-list & tick option has its own popup tooltip, and there is an incomplete prototype Help & Advice page.
• Birth & Death Dates use estimation algorithms borrowed from my other Plugins.
  (Beware that pre-Birth Dated facts may upset the estimation of Birth Date.)
• The cleaned Surname has changed from Living to Private.
• The 'For any Note/Media/Source/other fields' options apply everywhere including Names & Facts.

[ATTACHMENT deleted as now in Plugin Store.]

[BillH]

Mike,

I will run this and look at the output, but first a couple of thoughts. I do like the new look, order, tool tips, etc. Also, thank you for adding a Help button... when it is complete, this will be very helpful as well. In looking at the options themselves, I find some easy to understand, but a few confuse me.

1. Is this the correct way to interpret #2?

If the 'Living' flag is set then ... only/even if ((there is no death date and born after ...) or ( where no birth date exists then ...)) and (where any estimated dates use the ...)? If that is the case you might indent the "or where no birth date exists then" wording to make it a little more clear. i guess all the and's and or's are confusing me.

2. I don't understand the "If clean 'Living' rules apply to Spouse or Parent" option. It sounds like if this is set to "do nothing" then if you are going to clean the spouse or parent you are not going to apply the rules to the individual in question but if you pick the other option you are going to? Is that right?

3. I need help with the Note/Media/Source/Other options. You are going to either remove the local field and the links or remove the local field and the record. What record? For Note, you are deleting the local note field and deleting note record? I don't have any Note records, but I have lots of local notes. For Media, are you deleting the local media field (what's that?) and deleting the media record? etc.

Sorry, I am so confused. I guess I'm getting a bit dense in my senior days and it has to be really simply put to get me to understand anymore.

Thanks

[BillH]

Mike,

Here are some additional thoughts after running the plugin. These are the 4 individuals from my earlier exmples.

1: ? /Gauthier/ (ID 14738 ) Is still being forced to living. I was wrong before. This is a female. She has no information other than just a name. She has no children or parents. She has a husband who is dead (1904 - 1987). Why is she being forced to living? Is it because based on her husband's birth year 0f 1904 she might have been born after 1910?

2: Patricia H Henshaw (ID 5268) is still being forced to living. She has no other information. She has no husband and no children. Her father is dead (1892-1980) and her mother is dead (1896-1969). Why is she being forced to living? Is it because based on her parents birth dates she might have been born after 1910?

3: Oliver Strom (ID 3627) and his child are now being forced to living whereas before they weren't. This is what my method did and what I would expect.

4: Virgil William Niewedde is not longer being forced to living. He was born 14 Feb 1916 and died "bef 12 Sep 1966". This is what I would expect.

Overall it is looking really good. I don't see any obvious issues. No dead people are forced to living that I see.

Here are the options I used.

image1.jpg (211.66 KiB) Viewed 12037 times

Thanks!

[tatewise]

Regarding the forced to Living :-

1: I have created an equivalent couple with your Rules and she is NOT forced to Living. So I suspect there must be something else in her details. Have you checked on her All tab, although I cannot think what might be the cause? Does her husband have any relatives?

2: Yes, her estimated Birth Date will be based on her parent's Birth Dates and Tools > Preferences > Estimates for Mother's/Father's age at childbirth. So with those at defaults it gives midway of 1896+(12+48)/2=1926 from Mother and 1892+(12+70)/2=1933 from Father.

Regarding the Plugin Rules :-

1. If the or where no birth date exists then said or there is no death date nor birth date then would that be clear?

2. I agree the do nothing is misleading, and should say use normal 'Living' flag rules above.

3. I agree these need better wording.
remove local <items> & Links means delete local items, or if linked to records then unlink the records.
remove local <items> & Records means do all the above and delete any linked records even if linked to other items.
BTW: You may not use them, but you can have Local Media Objects that are NOT linked to a Media Record, but link directly to the Media File.
In the same way there are Local Notes and linked Note Records.
Also local Source Notes as well as all the local Citation fields, and the usual linked Source Records.

[BillH]

Mike,

In regards to the forced to living,

1) She has only name, sex, spouse family... nothing else. The husband does have siblings. They were born between 1902 and 1918. They are all dead except for one born in 1907. I'm not really worried about his person or the other similar ones. They are all close enough to being born in 1910 or after so having them marked as living is fine. I was just curious as to why.

2) That explains it.

As for the plugin rules,

1) That is a clearer for this option.

I am still confused as to how the three options under "If the 'Living' flag is NOT set then" fit together.

Are they all mutually exclusive (OR's) or are two of them linked (AND's)?
Is it 1 or 2 or 3?
1 or (2 and 3)?
(1 or 2) and 3?

2) I think this is clearer, but was wondering if this would be more clear... "do not apply the clean 'Living' rules below".

But that isn't exactly what was confusing me. I'm confused by exactly what the option text itself says. I'm guessing that you mean "use the following Clean 'Living' Person Rules on an individual if either their spouse or parent qualifies for these rules even if they themselves do not qualify for these rules".

If that is the case, it is too long to be the option text, but maybe you could incorporate something like that into the Help.

3) It is true I don't have any of those.

So, let's say I have a birth event with a local note.

The first option will delete the local note, un-link it from any records that are linked to it (the birth event and anything else), and leave the birth event and anything else it was linked to?

The second option will delete the local note, un-link it from any records that are linked to it (the birth event and anything else), and delete the birth event and anything else it was linked to linked to?

Is that what you mean? Why would you want to use the second option. Wouldn't this potentially delete the note from other individuals who aren't living as well? Wouldn't you also end up potentially deleting events that you really didn't want deleted? Sounds dangerous.

I am probably just still really confused on what the second option really does.

Thanks

[tatewise]

Try attached Clean Living Persons Plugin Version 1.2.8 Date 28 Feb 2017.
Mostly just a change to the options dialogue wording to make it clearer.

Regarding forced living:

1) The Result Set of Forced Living Individual now shows Birth Year that may help diagnosis for her.
Try with estimated dates option set to earliest and then latest to see what Birth Year is in Result Set.
Are you sure the where no birth date exists option is do nothing?
if set to apply clean 'Living' rules then the Birth Year will always equal the born before cut off year.

Regarding Plugin rules:

1) If the 'Living' flag is NOT set then three rules should be much clearer.

2) The spouse or parent option wording is much clearer along the lines you suggested.

3) Removing Note/Media/Source fields wording revised.
remove all fields and unlink records deletes local data & links, but does not affect the records linked to, but may get deleted later in Unused Linked Record Rules.
remove all fields and delete records deletes local data & links, and deletes the linked records, even if linked to other non-cleaned fields, as requested by David.

[ATTACHMENT deleted as now in Plugin Store.]

[BillH]

Mike,

Regarding forced living:

1) The where no birth date exists option is definitely set to do nothing. I ran the plugin with all date options and these are her birth dates shown in the result set.

default date: 1925
earliest possible date: she doesn't get cleaned so not in result set
midway date: 1925
latest possible date: 1959

I think I understand what the last three are, but what is default date?

Regarding Plugin rules:

All the options are much clearer. Of course some of them are very close to what I suggested so it makes sense that I would understand them. We'll have to see if other people find them understandable or not.

I have another question. This is not unique to this plugin. Is there any way to get the plugin window to pop-up on the same monitor that FH is currently on? I have three monitors and FH is never open on my primary monitor, but when I start a lot of plugins, they insist on being on the primary monitor.

Thanks!

[tatewise]

default date refers to the default setting for the =EstimateBirth/DeathDate() functions, which is MID and so the same as midwday date. I think that needs changing to get rid of default date as it means nothing to end user.

So her latest estimated birth date is 1959, with midway date 1925, which makes earliest date 1891.
Those dates must be derived from some relatives dates, facts & relationships, combined with Tools > Preferences > Estimates.
I would like to discover what is going on in case it reveals a problem.
It is a very small tree, so could you possibly post an exported Gedcom of just that family group?

I am glad the options are much clearer. Could you also review the tooltip popups, and suggest what might need to be said in the Help & Advice page. If you could various Plugin options to see if they behave sensibly it would be useful.

I am not sure what determines which monitor would be used. Some Plugin windows are created via the FH API and some via the Lua IUP Portable User Interface. Could you list which Plugin windows use which monitor and I might be able to work out the cause.

[BillH]

Mike,

I'll take a look at the other things you mentions, but here is the small gedcom containing ? Gauthier and her close family.

Thanks

[ATTACHMENT deleted as no longer relevant.]

[BillH]

Mike,

As far as the tool tips themselves go, they seem pretty clear. I do have a one thought. The ignore relatives and do nothing tooltip says it applies primarily where current person has died . So it could apply where the current person has not died? Under what conditions? This should probably be spelled out in the help. All the others seem fine to me.

I'm not sure I understand why there is an option on If the 'Living' record flag is set then to ignore the flag and do nothing. This appears to mean that you want to skip cleaning on any individual that already has its living flag set. I can't imaging why you would be running a plugin called Clean Living Persons if you wouldn't want to clean a person that is living.

After looking at it a bit more, I guess I am still a little confused on the options under If the 'Living' record flag is NOT set then. The option with the cut-off date says Apply the 3) Clean 'Living' rules only if there is no death date and birth date is after but then you give another way to apply the rules. Would it be better if that item and the next item said something like the following?

Apply the 3) Clean 'Living' rules if there is no death date and birth date is after 1910
Apply the 3) Clean 'Living' rules if there is no death date and no birth date

As for the Help & Advice, I think really just give good long non-technical instructions on how to use each option. What exactly will each option for each item end up removing? If possible avoid using terms that only someone who really understands the ins and outs of GEDCOM or FH file structure would know. Give examples when possible,especially for the For any <Note/Media/Source> fields and linked records options.

I've run some tests using the options I don't normally use and the results seem to be what I'd expect. I'm not sure I understand exactly how the two options for Apply the 3) Clean 'Living' rules if there is no death date and no birth date work, but the apply rule does clean about 600 more people on my database and they all seem appropriate.

After further testing I am finding that all plugins insist on using the primary monitor rather than the one that FH is running on. The Help & Advice for most use the primary monitor, except for Clean living Persons which uses the monitor FH is running on which is odd. It would be nice if this could be changed so that they use the FH monitor if possible.

Thanks

[tatewise]

I have analysed your Gedcom and she is being cleaned because of her Marriage Date you forgot to mention.
I suspect your Tools > Preferences > Estimates sets Woman's age at marriage between 14 and 82.
So for Marriage in 1973 her estimated birth lies between 1973-82=1891 and 1973-14=1959 with midway of 1925.

The If 3) Clean 'Living' rules apply to spouse or parent option can apply when current person is alive where either of the preceding no death date rules inhibit cleaning.
Either the born after cut off year is large, or the no death date and no birth date is set to do nothing.
They have no death date, so may be alive, but would NOT be cleaned, except for their spouse or parent being cleaned.

The ignore Living flag option was to allow for a database where the Living flag is unreliable.
On reflection, the option might be better as apply death and birth rules below or perhaps treat as if 'Living' flag were NOT set so that cleaning uses the rules below instead of the flag.

I agree the word only is misleading and will be removed.
If the 2nd rule is Apply the 3) Clean 'Living' rules if there is no death date and no birth date then its drop-list values would be Yes & No or perhaps a tick box. Is that what you implied?

As it stands, the Help & Advice box is a simple plain text window limited by the size of the screen and has no scroll bars.
So long non-technical instructions may be difficult to present in an easily understood style.
I was thinking of providing a few key explanations such as to use the Export > GEDCOM File command beforehand.
Then refer the user to the FHUG Knowledge Base for Exporting a Family Tree with/without Media and in particular the Export to Website Without Sensitive Data section, where the long non-technical instructions could be added and updated without needing to re-publish the Plugin in the Plugin Store.

The if there is no death date and no birth date options are either do nothing or apply cleaning.
So I am not clear what is difficult to understand, except I now see that ignore the dates and do nothing should say ignore that case and do nothing because there are no dates to ignore!

The vast majority of user dialogue windows for Plugins use the Lua IUP Portable User Interface library, which runs as a separate task and presumably defaults to the primary monitor. Not sure if it is even possible to make it use another monitor. Clean living Persons uses the fhMessageBox() API that is part of FH and thus uses the FH monitor. This will apply in my and other Plugins when for example there are no Result Set details, or some other minor status needs reporting, and it is easier to use the fhMessageBox() API than build a window using Lua IUP. An example is my Split Emigration and Immigration 2 Place Events Plugin if you run it again after splitting, such that no more need splitting and there is no Result Set.

[BillH]

Mike,

I mentioned that she was married, but I did forget to mention the date didn't I. To be honest I didn't even really pay attention to the date. You were close, I have the date range set to 14 to 80. Based on this marriage date it does make sense for her to be marked as living.

Thanks for the explanation on the If 3) Clean 'Living' rules apply to spouse or parent option. This is the kind of info that would be good in the help or in the KB somewhere.

For ignore Living flag option I like both of your re-wordings. The first is consistent with what you say if the flag is not set. The second makes it clear what you are doing. I think either is fine. Just to be sure I understand, if you were to make this change the two options for this item would be:

1) apply death and birth rules below or treat as if 'Living' flag were NOT set
2) apply 3) clean 'Living' rules below

Is that correct? Actually, after seeing these two options like this, maybe treat as if 'Living' flag were NOT set would be less confusing. I'll leave it to you, I think either would work fine.

I hadn't really thought through what the options would be If you used Apply the 3) Clean 'Living' rules if there is no death date and no birth date. I think either a drop-list with values Yes & No or a tick box would be fine. I think it would just be aesthetics. Whichever you like would be great.

I think your idea of a reference to the KB would be fine. As long as there is someplace to get help.

On my comment about if there is no death date and no birth date, the values are clear (especially with the correction you mentioned). I didn't explain very well. As I mentioned in an earlier post in this thread, I never used this option in Jane's version of the plugin as it scared me. It sounds like what is going to happen is that anyone without a birth and death date is going to get cleaned even if they lived in the 1600's. I now realize that this isn't the case. Your tool tip makes it clear that it really refers to not having "recorded" birth and death dates or "estimated" birth or death dates. So I guess my question really is this. After going through all the logic to determine "estimated" birth and death dates, what conditions would result in a person still not having any "estimated" dates. In other words, who is really getting cleaned as a result of this option. This would be something that could be documented in the KB.

You say that Clean living Persons uses the fhMessageBox() API that is part of FH and thus uses the FH monitor. This is not what I am seeing. The plugin still uses my primary monitor even though FH is not running on that monitor. In any case, it is not a big deal. It was really a nice to have if it could be done.

Thanks

[tatewise]

That all sound OK. When I have adjusted those options I will post an updated Plugin.

The conditions that would lead to no death date and no birth date are just like your Gauthier example if there were no Marriage Date. There simply is not enough to even estimate any dates.

What I meant was that Clean living Persons uses the fhMessageBox() API for the Help & Advice page, which I think you said appears on the FH monitor. (It also uses it for the initial warning if run in Project mode, but you won't be seeing that.)
The main options dialogue uses the Lua IUP which will appear on the primary monitor.

[BillH]

Mike,

Thanks for the explanations... makes sense. Also thanks for all the hard work on the plugin.

[tatewise]

I think the attached Clean Living Persons Plugin Version 1.2.9 Date 03 Mar 2017 should be near the final version.

It incorporates much of what we have discussed but the Help & Advice button now directs to the FHUG Plugin Help page plugins:help:clean_living_people:clean_living_people|> Clean Living People Plugin that can be easily updated with any extra advice.

Regarding Plugin windows, for many of my Plugins with sticky settings, have you tried dragging them onto your FH monitor?
They are designed to remember their position and should stay where you put them.
(It does not work for this Clean Living Persons Plugin.)

[ATTACHMENT deleted as now in Plugin Store.]

[BillH]

Mike,

I really like it. I think all the options are worded clearly and the way you have them organized now is good. I think linking the help to the KB will be fine. I will read through it in a bit and see if I see anything that might benefit from some additional comments or is confusing.

On first glance, I only see one situation that I can't quite explain. I have attached the options I am specifying. The situation I'm seeing is where I already have the Living flag set on my database and I'm telling the plugin to treat as if 'Living' flag is NOT set. That fact may not be relavent to what is going on.

For about 20 individuals, they are not getting changed to Private. They all have no birth or death dates. They are all married to folks that were born after 1910. In all cases their spouse is either dead or marked as Private. Here is one example with a dead spouse.

image2.jpg (103.1 KiB) Viewed 11893 times

I would have thought Deloris (18462) might have been treated as living since her husband was born after 1910 and she was married in 1948.

Here is one example of a spouse who was living.

image3.jpg (101.41 KiB) Viewed 11893 times

I would have thought Tracy (11130) might have been treated as living since his wife was born after 1910 and they were married in 1941. His wife Joyce does get changed to Private.

Both of these individuals have very little to go on, but I thought maybe their marriage date and/or spouses birth dates might trigger them to be considered living.

As for the plugin windows. If I move the window to the FH monitor, close the plugin, and then reopen the plugin, it opens on the FH monitor. However, if I close FH and then reopen FH, the plugin opens on the primary monitor again.

Thanks!

[tatewise]

So the Living flag is disregarded and the death and birth rules applied.

The spouse birth dates have little influence over estimating their partners birth date.
Your Tools > Preferences > Estimates for age at marriage I assume is 14 to 80 for both sexes.
The mid-point is 47 so any estimated Birth Date is 47 years before Marriage Date if no better data is available.
Thus Deloris was born 1948 - 47 = 1901 that is before 1910.
Also Tracy was born 1941 - 47 = 1894 that is before 1910.

If you change to use latest estimated dates then more people will be included as 'Living' and will include Deloris b. 1948-14=1934 and Tracy b. 1941-14=1927.

If you could add husband & wife Ages to those Marriage Events it would make all the difference.

I don't understand what is happening with the Plugin window. If it can remember the FH monitor position between runs, then I cannot explain why it forgets, just because FH gets closed and reopened. The Plugin will be using its same 'sticky' saved settings. Which of my Plugins have you tried? If instead of moving to the FH monitor you just move the Plugin window to top left corner of primary monitor, does that position always get remembered?

[BillH]

Mike,

OK... that explains it. I may play around a bit with using the other date options. Unfortunately I don't have any age info for the marriages. I only enter ages if they are stated in the source documents.

OK... this is really weird. I re-booted (for an unrelated reason) and now it seems to remember the window locations just fine... even if I close and restart FH. So... no idea what I was seeing earlier.

Thanks

[BillH]

Mike,

Are you planning on publishing the Clean Ancestry Fields plugin or is this a one-off for me?

Would it be hard to change it to get rid of any %INDI.FAMC[x].PEDI[x]% fields? Somehow I missed these before, but they also create the type of records that Ancestry handles so poorly. If it isn't too hard to do, I'd be happy to make the changes to my copy of the plugin (with a little guidance from you as to what to change).

Or if it makes sense to do so, would you add a Remove entirely option to the Extra Options tab of the Export Gedcom File plugin?

Thanks

[tatewise]

At this stage, unless somebody else shows any interest, I was expecting the Clean Ancestry Fields plugin to be just yours.

As well as %INDI.FAMC[x].PEDI[x]% there are also %INDI.FAMC[x]._PEDI[x]% custom Pedigree fields.
Currently, the former as standard Gedcom are not changed by Export Gedcom File, and the latter are converted via a local Record Note to a synthetic Ω custom Event for Ancestry.
Is it both or just one of those tags that need removing?

[BillH]

Mike,

I didn't realize there were both. Actually, I think it is only the one that creates the synthetic fact that needs removing. If you let me know what needs changing, I'd be happy to try to update the plugin myself. Would I just add _PEDI to the FAM={...} statement to make it FAM={"SOUR";"SOUR2";"_PEDI";};

Thanks