* FH-RM-Ancestry Deceased/Living issue

[jelv]

I'm having an issue with a tree I've uploaded to Ancestry using the Family Historian - RootsMagic - Ancestry Sync plugin. Everybody in the uploaded tree is shown as being deceased which means that everyone would be visible if I made the tree public on Ancestry.

The Ancestry support site says

If no death information is provided, people under 100 years old are considered to be living.

so I'm puzzled as to where the deceased status is coming from.

As a test on Ancestry I exported the tree, then changed the status of an individual to Living and exported the tree again. Comparing the exported Gedcom files there is no difference.

Setting the status of an individual in Rootsmagic to Living, I'm having trouble getting TreeShare to update it on Ancestry.

Any pointers please?

[Mark1834]

Hi John - Living/Deceased status on both Ancestry and RM can be a bit erratic, as they seem to make up the rules as they go along! I've had instances of entries on Ancestry without definite lifespan dates being shown as living, even though they were the parents of children born hundreds of years ago!

The plugin takes the Living flag on FH as the definitive source of information, and makes no other assumptions, even if the birth was relatively recent. As part of the Update step in the plugin, it forces all Living flags in RM to match the values in FH. Once they are all correct in RM, they should sync with Ancestry correctly.

By default, the plugin does not export individuals where the Living flag is set, but this can be changed in the plugin options.

Does that explain what you are seeing, or do you have an example of where that behaviour is not being followed?

[jelv]

Thanks Mark, it does explain what I am seeing.

My problem was that as until now I had no need to use the living flag (the 100 year rule was sufficient for my purposes), none of my individuals had the Living flag set.

I've now found Flag Living and used that so will have a go at a new sync.

There's no mention of your use of the Living flag in Synchronizing Your Tree With Ancestry to Exploit Hints - I would suggest the KB article is updated to include the information you've given here (and a link to Flag Living) to help others avoid the difficulties I've had.

[Mark1834]

I would recommend having the Living and Private flags displayed in both the FH Property Box and the Records Window, as that makes it very easy to spot and rectify any incorrect values. The KB includes how it handles individuals marked as Living, and how the flag is synced with RM, so it is there.

[jelv]

Living/Deceased status on both Ancestry and RM can be a bit erratic, as they seem to make up the rules as they go along!

I'm having major issues with this!

I created a new test database in RootsMagic with just a couple of individuals, both with the Living flag set. I then used Tree Share to create a new tree on Ancestry. They were both shown as deceased! From what I've found it appears that the Living status will only get updated on Ancestry if some other change is made to the individual.

Edit: ... and even then it is likely to fail.

[Mark1834]

It's certainly the case that the Living flag does not seem to be included in the RM TreeShare record comparison. Records are shown as equivalent if that is the only difference, so there is no way to force an update. That's a defect in RM/ANC that we can't do anything about.

Remember that the core purpose of the plugin is to provide a consistent sanitised extract of your tree on Ancestry for hint maintenance. Personally, I don't think that requires living people to be included, but the option to do so was added later following a user request. The plugin ensures that FH and RM are fully synced, but it can't control defects in TreeShare. I believe that most of the problems arise from the initial upload, and subsequent updates are more reliable, but it is "user beware", I'm afraid.

My own preference is to keep all living and private individuals off of Ancestry completely. Marking them as private is purely a fig leaf - details can usually be reconstructed very easily from information relating to deceased relatives (certainly for UK trees, at least).

[jelv]

What's puzzling is that if in RootsMagic you edit a person and change only the living status, then run TreeShare it shows the person as being changed, but there is no arrow to make the changes and all you can do is use the X to Mark as "not changed".

I'm waiting for approval to join both the RootsMagic Community forums and the RootsMagic Users Facebook group and will be asking about the issue. If it's not going to update the Living status it shouldn't be showing the person as being changed when you run TreeShare if the Living status is all that has changed.

By the way I've spotted minor issue with your plugin. In function UpdateRMTimeStamps you are rounding the modified date/time to always be midnight by using floor division (//) which returns an integer. Line 1724 should be

Code: Select all

local today = os.time()/86400 + 25569

I'd assumed TreeShare worked by comparing modified times, but correcting this didn't improve things.

[Mark1834]

Yes, it's frustrating having to deal with clear limitations in other apps. There is a significant amount of code in the plugin that does nothing more than check and correct potential errors and omissions in RM importing and merging! FTM seems to offer a much more powerful (and hopefully more reliable) ANC sync, as it is fully automatic, but MacKiev have made the business decision to encrypt their database, making it inaccessible to anything other than FTM.

Good spot on the code, thanks, but it's actually deliberate. The plugin was developed originally for RM7, which only stores the date a record was last updated, not the time as well, so needs an integer value. RM8 stores the time, but as far as I can see, still only reports the date in the UI, so I've left it as date only for both versions for consistency. As you say, it does not seem to affect how TreeShare functions.

[fhtess65]

I'm a member of both groups, even though I no longer use RM as my main software - will watch to see how this develops in those forums. TreeShare can be quite a contentious issue, so don't be surprised by some of the answers you receive.

Teresa

<SNIP>

I'm waiting for approval to join both the RootsMagic Community forums and the RootsMagic Users Facebook group and will be asking about the issue. If it's not going to update the Living status it shouldn't be showing the person as being changed when you run TreeShare if the Living status is all that has changed.

<SNIP>

[jelv]

Good spot on the code, thanks, but it's actually deliberate. The plugin was developed originally for RM7, which only stores the date a record was last updated, not the time as well, so needs an integer value. RM8 stores the time, but as far as I can see, still only reports the date in the UI, so I've left it as date only for both versions for consistency. As you say, it does not seem to affect how TreeShare functions.

Is the field name the same on RM7? The info. I found suggests it wasn't.

https://sqlitetoolsforrootsmagic.com/date-last-edited/

[Mark1834]

EditDate in RM7, UTCModDate in RM8. RM sensibly kept the same table structure from RM7 to RM8 where it was forwards compatible, but this was a new field definition, so they gave it a new name - again, very sensible and logical.

[jelv]

Could I suggest this code modification which would mean it works correctly for both RM7 and RM8.

Code: Select all

	-- get today's date and convert to RM format (days since 31 Dec 1899)

	local today = os.time()//86400 + 25569 
	local todaytime = os.time()/86400 + 25569 
	local S = fhLoadTextFile(DataFile)

	for UID in S:gmatch('[^\r\n]+') do
		local SQL
		if FileName:match('.rmtree$') then
			SQL = 'UPDATE PersonTable SET UTCModDate = ' .. todaytime .. ' WHERE UniqueID = "' .. UID .. '"'
		elseif FileName:match('.rmgc$') then
			SQL = 'UPDATE PersonTable SET EditDate = ' .. today .. ' WHERE UniqueID = "' .. UID .. '"'
		end
		database:execute(SQL)
	end

[Mark1834]

If it is changed, it needs to be as part of a more comprehensive review. I'm not confident that I fully understand how the field is used across all of RM7 and RM8, as all this direct access stuff is reverse engineered by enthusiastic users, not supported and documented by RM Inc. Unlike FH, RM tracks update stamps for each specific event (but doesn't display them in the UI), so should I update these as well? Should it be as integer (date) or float (date/time)? Should stamps be converted from local time to UTC?

At the moment, it's not a priority, as keeping everything in a basic RM7-compatible format and just changing the stamp for the whole record seems to work. UTC might be worth investigating if a non-UK user reports mismatched dates, but even that would probably just be cosmetic and not affect function.

However, plugins are open source, and a second pair of eyes is always useful. If you want to investigate yourself in detail, I'll be happy to incorporate any improved functionality in the master copy if it makes a practical difference.

[jelv]

It's certainly the case that the Living flag does not seem to be included in the RM TreeShare record comparison. Records are shown as equivalent if that is the only difference, so there is no way to force an update. That's a defect in RM/ANC that we can't do anything about.

I'm tending towards thinking that it compares the UTCModDate in AncestryTable with the UTCModDate in PersonTable (and records linked to each person), then it compares the data in the fields where records have a changed time. If you change something, save it, change it back and save TreeShare says it's a changed record but there's nothing shown as being different and you have to hit the X.

From the query I've raised on the RM Community it seems to be a bug that it's not handing Living at all - I've raised a ticket with RM.

If I'm right then we should set the proper date time in RM8 if it is to work correctly. Whether that needs to be corrected to UTC would depend on whether it checks for any difference or is looking for latter - for the former it wouldn't matter, for the latter it would.

Remember that the core purpose of the plugin is to provide a consistent sanitised extract of your tree on Ancestry for hint maintenance. Personally, I don't think that requires living people to be included, but the option to do so was added later following a user request. The plugin ensures that FH and RM are fully synced, but it can't control defects in TreeShare. I believe that most of the problems arise from the initial upload, and subsequent updates are more reliable, but it is "user beware", I'm afraid.

My own preference is to keep all living and private individuals off of Ancestry completely. Marking them as private is purely a fig leaf - details can usually be reconstructed very easily from information relating to deceased relatives (certainly for UK trees, at least).

I'm putting my tree on Ancestry for multiple purposes. Yes to get hints, but also to share with relatives I've contacted on Ancestry and possibly to make public if that is going to make it more likely distant relations will contact me as a result of my having done a DNA test. Regarding hints, there may well be hints for living relatives so I'd be inclined to to include them.

I'm therefore thinking I need more than one tree on Ancestry. One which is private and not included in Ancestry indexes and has everybody and I never share it so whether Living individuals are included wouldn't matter. I'd use this just for getting the hints. Then one or more trees which only have deceased individuals (probably one for my Father's relatives and one for my Mother's and same for my wife).

My thinking is that I'd initially sync with the everyone tree on RM & Ancestry which would mean everyone in my tree has a UID. Then copy the FH project and use the Split Tree Helper and create the other tree(s) in RM and Ancestry with Living excluded. For subsequent updates of the other tree(s) I could delete the the FH project, copy and split again (relying on the UIDs in the new FH project being the same as the last time which should keep your plugin happy). I suspect I'd need to preserve FH RM Ancestry Sync.cfg across that process.

Can you see any flaws in this plan?

[jelv]

I've had a response from RM Support:

Yes, this issue happened in our last update. It will be resolved in our next update but I cannot tell when that will be released. What you could do is create a group of everyone Living in RM and then in TreeShare if you have nothing to add to them click on Show on Ancestry and change their Living status there. If you did have something to add to them from RM to Ancestry it would update their living flag on Ancestry.

When creating the group click on Mark and select Living. In Ancestry TreeShare select the group by clicking on the box "Show Everyone" at the top.
http://wiki.rootsmagic.com/wiki/RootsMagic_8:Groups

[Mark1834]

I'm tending towards thinking that it compares the UTCModDate in AncestryTable with the UTCModDate in PersonTable (and records linked to each person), then it compares the data in the fields where records have a changed time. If you change something, save it, change it back and save TreeShare says it's a changed record but there's nothing shown as being different and you have to hit the X.

That's certainly possible. I have a small test dataset that I use to explore how TreeShare works, and if I change the value of an event directly using SQLite Expert Personal (so without updating the timestamp), TreeShare does not highlight it as a changed person when I load it back into RM.

One odd thing that I have noticed however is that if I make a change directly in RM, the value stored in UTCModDate in PersonTable can be either an integer or a float! I've changed five individuals either yesterday or today, and three have integer only values stored, while two use the full float value . Does it depend on the type of change made, I wonder? That could make it very complicated to work out what is going on.

I must admit, I don't take any notice of which individuals RM thinks have changed when I update Ancestry, as I just work through the alphabetical list of changed individuals from the plugin Research Note report. That's the definitive list, with no false positives or omissions.

Thinking further about the update stamp change that the plugin makes, it is reading the Epoch time in seconds and converting that to a day number, so it is in UTC already. The RM UI will be converting it to the current date in local time, so it's not something we have to worry about.

[jelv]

One odd thing that I have noticed however is that if I make a change directly in RM, the value stored in UTCModDate in PersonTable can be either an integer or a float! I've changed five individuals either yesterday or today, and three have integer only values stored, while two use the full float value . Does it depend on the type of change made, I wonder? That could make it very complicated to work out what is going on.

I wonder if this explains some of the issues some people are having with TreeShare - when they changed from EditDate to UTCModDate they did a global search and replace but failed to change the calculations in some places?

I must say that he more I do with RM the happier I am that I chose FH as my primary software!

[jelv]

8.2.6 - 28 OCT 2022

• Fixed: Some living people were marked as deceased when uploading to Ancestry

But from a quick look it will only apply to individuals who have changes, no correction to all the others wrongly uploaded to Ancestry as deceased.

[Mark1834]

I find it frustrating that I haven't found a simple way to filter the person list in the Ancestry tree display to show only those marked as living. It's trivially easy in both the Android and iOS apps, but I can't find it in the browser view of a tree. Probably hidden in plain sight somewhere...

I did a large update of my online tree yesterday, and before running the plugin I cleared the "changed" list in TreeShare to see what would show up afterwards. Although there were updates to several dozen existing entries, only a handful showed up as "changed" in RM after updating. It doesn't seem to spot changes through merging, just as it doesn't update the record timestamp, so I'll carry on using the definitive plugin list to guide the update, rather than rely on flaky RM programming to list changes.

[jelv]

Probably hidden in plain sight somewhere...

If it is, I can't find it either!