* FH7 to FTM2019 File Merge Issue

[David Potter]

Hi All
I have tried the FTM2019 Facebook Group to answer this and reported my issue to Mckiev. No joy. I'm hoping someone here does the same as I'm trying to do and has an answer to my issue.

When I export a GEDCOM file for my FH7 database into FTM2019 (with latest update) all goes well and people counts, etc match up nicely. I have used both the FH7 default export feature and Mike Tate's Export Plugin. Then when experimenting with the FTM2019 Merge File Feature and importing exactly the same GEDCOM file. I see a small number of people I need to review; and these make sense as to why they are described as needing review. I also see a large number of precise matches which I'm very happy about. Then I see over 300+ new people which I don't understand at all, even changing the Match Threshold to the maximum value of 1000 doesn't solve the problem.

So when importing the exact same file twice I don't understand why the process should mark a large number of people as NEW.
I use Custom Id's throughout FH7 and these match in value with the FH record ID, I also use the FH7 Unique Identifier feature but I not sure FTM2019 makes use of this.

Any help or suggestions very welcome.

[Mark1834]

It doesn’t answer your question directly, but a very similar thing happens in both RM7 and RM8. Perhaps it demonstrates that comparing individuals on anything other than UniqueID is unreliable...?

Personally, I wouldn’t touch FTM 2019 with a barge pole - I’m not at all comfortable with an app encrypting my data so I can’t audit what is happening under the hood.

[tatewise]

As Mark says, this is not an unusual problem.
For comparison, what happens if you ask FH7 to merge those two identical GEDCOM files?
You say some records need review and is understandable. Please explain.
I have no knowledge of FTM but would intuitively guess reducing the threshold for a match would improve record matches.

[David Potter]

Thank you both for your comments.

@Mike - the records marked as review are Surname only individuals with a sequential ID number. I create these Individuals where a 1911 Census reports number of children born and died. If I cannot account for difference by real named persons I create a surname only individual(s). So if I cannot account for say two children I create for example:

Potter // ID 34
Potter // ID 35

I will try the FH merge of the same file and see what FH7 does with the Individual count.

Thank you both.

[David Potter]

Um, not quite sure what is happening here. I imported the GEDCOM i had created for use in FTM into FH as a first step to evaluating what FH would make of the file Merge process. This first step went through the validation process and produced two log files (see attached). Looking more closely at the Validation Log the import process has removed a large number of family links along with a few other types of link. Taking one example where the report states ID 144 is not listed in the family list of children and therefore 144 is removed from that family. That is simply not true I can see ID 144 in the family list in my Master Database that acts as the source for the Gedcom, so I'm not sure why this should cause such a problem in FH and whether or not this is contributing to the issue in FTM.

Secondly, I took a copy of the FH master DB Gedcom file and imported that into FTM to create a new DB there. Then attempted to Merge that same file in FTM, same problem occurs with large numbers of 'New' Individuals.

I have run the FH Validation process over the master DB and no issues are found!

[David Potter]

I forgot to mention that using the Match Threshold of both Min and Max does not significantly reduce the number of 'new' individuals suggested in FTM.

[David Potter]

Here are two screen shots of the before and after import of the Gedcom. Showing the Family links to children removed.

[Mark1834]

David, what happens if you follow the same process for the Sample Project? That may help to resolve which of the three common root causes (program error, user error, or data error) is to blame.

[tatewise]

I am away from my PC until the weekend and will look at this then, but it seems to be more of an FTM problem that can only be fixed by them.

[Mark1834]

Not sure that follows - David is saying that he exports a GEDCOM from his master database, then imports that back into FH as a new project. This drops a number of family links, due to inconsistencies in the GEDCOM file.

The normal structure of a GEDCOM file for individuals is to define the family/families where they are a child, and where they are a spouse. The family section of the file lists the husband, wife, and children of that family, so each link is defined from either direction. The error log indicates that one of these links is missing in a large number of cases, so it appears to be a GEDCOM export from FH error, and occurs with files that have never seen FTM.

Is that the correct interpretation of the problem?

[AdrianBruce]

... The error log indicates that one of these links is missing in a large number of cases, so it appears to be a GEDCOM export from FH error, and occurs with files that have never seen FTM. ...

Caveat - I'm not familiar with FTM or the processing done when producing a GEDCOM for FTM. But just a thought from me...

While the result is worrying at first glance in that it appears to involve issues with round-tripping a GEDCOM produced by FH out and then back in (if I interpret it correctly), could it be that the variety of GEDCOM produced by FH (and / or plug-in?) has been deliberately "altered" towards what FTM expects on an import and away from strict GEDCOM? And therefore it is liable to cause problems if imported into FH? Which is presumably expecting either straight GEDCOM or FTM export GEDCOM? (which may not match FTM import GEDCOM, of course...)

[Mark1834]

It begs an experiment. Three different GEDCOM exports: FH built-in with default settings, Mike’s plugin with default FTM settings, and the FH-RM-ANC sync GEDCOM export, which is plain vanilla GEDCOM with just basic data. Which of these files drop links when reimported?

[David Potter]

Thanks all for participating in this issue, much appreciated. Mark has nailed what appears to be the problem everything I have done to cause this has been internal to FH. I am not returning gedcom files from FTM to FH.

The suggestion to test with the Sample file is a good one to try to narrow down where the problem lies program, data, etc, I'll work on this shortly and report back. I will also import the sample file into FTM to see what happens there.

Thank you all very much.

[David Potter]

I will also test the three scenarios Mark mentions in his last post.

[victor]

I regularly update my details on FTM. FH is my default file and I work on that all the time. From time to time (several months apart) I copy FH to FTM. I don't care how it appears on FTM I just use the details for access to Ancestry
To me details on FH is all that matters

Victor

[David Potter]

Hi Victor, thanks for joining the post. When you say you copy to FTM do you create a new FTM project each time or do you in fact use the Merge > File feature that I'm having issues with?

[David Potter]

Here are the results of a number of tests following Mark's suggestions.

All tests were made using the Sample Project, one As Is and another after firstly running the Sample Project Reset feature from the Project Window.

I then copied the Sample Project from the FH Sample Project Folder and pasted the file to the Windows desktop. I then used the FH New Project button from the Project Window dialogue and chose to import a Gedcom file. Some missing media links were found and i chose to ignore this and select not to have FH search for the media. After importing both GEDCOM's according to the above scenarios the As Is and Reset versions of the Sample Project. The 'new' Sample Projects are created by FH but suffixed (2) and (3) to avoid overwriting the original Sample Project.

Both tests did not produce any Log Files. Furthermore, the Sample Project (3) Gedcom was sent to FTM, and then the same file Merged in FTM, Two New Individuals were identified. Still a reasonable issue considering there are only 104 Individuals in the Sample Project.

The next series of tests follow Mark's suggestion to use FH Export Gedcom (Default), Mike's Export Gedcom Plug-In (reset to defaults) for FTM2019 and Mark's FH > RM > Ancestry Plug-In.

The Sample Project (3) Gedcom from the first series of tests detailed above was used with the FH Export (Default) to create a new GEDCOM file named FH_Default, when this file is imported as a new Gedcom in FH no log files are created.

The same procedure but using Mike's Export Plug-In did create a log file but the contents are nearly all Info Only reporting data had been moved to Note fields. Nothing appears to be dropped.

The last test was to use Mark's Plug-In to which a log file was created and it contained one record whereby the Family/Child Link had been removed.

With the above findings I used the FH Export Gedcom (Default) to create a Gedcom using my Master File, chose new project and imported the Gedcom, No Log files were created, furthermore the Individual I mentioned yesterday retained his Family Links.. Lastly this file was sent to FTM whereby the same problem I reported yesterday occurred. Significant number of New Individuals are reported during the Merge process.
So apart from the discrepancies reported above the FTM Merge process appears flawed.

Both Log Files are attached. Exception Report is for Mike's Plug-In, Validation Report is for Mark's Plug-In.
I do hope the above makes sense.

Thanks for your help.

[victor]

Hi David
In reply to your question. Only just read your message
When you say you copy to FTM do you create a new FTM project each time or do you in fact use the Merge > File feature that I'm having issues with?
No I do not create a new project. The only project I have is on FH and what I copy, from time to time, from FH to FTM is just to provide links to Ancestry.
I never work on my tree on FTM only on FH

Victor

[David Potter]

Hi Victor
Thank you for your reply. Can you please answer the following questions:

1) What version of FTM are you using?
2 ) If you are not using the FTM merge file feature how do you 'copy' your FH data into FTM. Do you copy the entire FH file or do you copy on an individual basis one person at a time?
3) Does the count of Individuals in FTM match what you have in FH?

Can you please screenshot the function in FTM that you use to import your FH file.

Thank you.

[victor]

I have FTM 2019
I just copy the FH details and never merge anything on FTM
On the left on FTM is a list of files called tre browser.
The latest tree I have copied from FH is listed. I just prompt this. Never merge anything. No point in merging as FH is the only tree I work on. I only use this to provide a link to trees on Ancestry ensuring this tree is sync with ancestry.
The list of names on the FTM tree show a green leaf showing there is a source on Ancestry. I prompt this leaf and it opens to a list of other peoples trees which I then check
The count of individuals on;y matches the particular tree I bhave copied. If I have added names on FH since the copy it doesn't increase the number

[David Potter]

Hi Victor
Thanks for the reply. I understand how you are using FTM now. You are creating a new tree with each new 'copy'. Whereas I'm trying to update one tree only.

Thanks again

[victor]

No I am not creating a new tree on FTM. The only tree I work on is in FH
From time to time I copy the latest tree from FH to FTM.
This is only for me to connect the names on Ancestry to see what is available on there. If I find anything new I don't add this to FTM and only FH. Any images I save on my computer and add them to the name on FH
On my computer I have a file in FH Projects called family information. This is sub divided into surnames on my tree. Not only surnames but also a census file and certificate files.
I have added a sub file for each surname where I save photos of individuals which are in turn added to FH media. The same applies to certificates (separate files for BMD) and census (separate files for census years and countries)
I do not add any of this information to my tree on FTM

[David Potter]

Hi Victor
I'm still confused how your work flow looks when you 'copy' your FH file to FTM. As there is no copy function in FTM, at least not that I'm aware of.

Would you kindly list out the steps you take to handle this 'copy' process.
EG, In FH...
1 I do this...

EG, In FTM
1 In FTM choose Menu > File > etc...
2 Do this...
3 Do that...

Many thanks
David

[victor]

My tree on FH is backed up into a file. Lets call it VLM1
On FTM on the left thre is a column heaaded 'Getting Started)
This lists a number of lines where one says
'Import on Existing tree'
The exisitng tree is in the file mentioned above so prompt the arrow and find the file
Below this are listed trees that have been imported in the past. No matter how manyb trees have been imported the list remains until you delete it.
Once you have imported a tree like VLM1 mentioned above that appears on the list
Once you have imported a tree that opens up when you open FTM
Hope this helps

[David Potter]

Thank you Victor.
As I suspected you are in fact IMPORTING your FH database into FTM. Each time you do this you are creating a NEW tree as far as FTM is concerned. So you have a number trees in FTM that if uploaded to Ancestry will create duplicate hints across the trees you have sitting in FTM.

Whereas my goal was to MERGE the FH database with one tree in FTM, sync to Ancestry and then I only have one source of hints to analyse.

That now removes the confusion - Thank You.