* Note Records with no Links (FTM>FH import)

[rob p]

I think this is probably (?) an Import issue, so I post here.
I recently imported from FTM2014.1. I checked and validated both the FTM export GEDCOM and the resultant FH GEDCOM. To “validate” the transfer, I opened both FTM and FH, and looked back-and-forth to see what went where, and insured nothing was “lost”. I easily cleaned up the few UDF’s.

But in my before-getting-back-to-research clean-up, I find an issue I can’t figure out: I have 100’s of Note Records that aren’t linked to any other data. In my transfer validation, these 'un-linked' notes DO appear in my FH file (not always where I expected, but none-the-less there).
I can find them in the GEDCOM as an OBJE- NOTE-CONT/CONC structure (by searching for the text). And the ‘@Nxxx@ NOTE’ record is there too, and is indeed not linked to anything.

Another confusing matter is that all of the other Note Records are linked to only one other record. The ones that would have been a candidate for a Note Record (used multiple times) are duplicated. Some are truly used only once.

I can’t make any sense as to why or how these “crazy” records occurred. Some of the same type of notes came over as linked Note Records, and some as ‘normal’ notes attached to the record that they relate to. Everything appears to have made the trip.

Why?/How? is just a curiosity question. I suspect the FTM-to-FH import. Does FH handle GEDCOM from FTM (as read in the header) uniquely? Would changing that to “Other” produce a different result? (I don’t really want to try that – I have a clean GEDCOM and no UDF’s!)

They don’t seem to cause any problems. But I’d like to clean my file of unnecessary baggage.
Can I safely just delete all of these Note Records with ‘0’ links??
Is there such a thing as a plug-in to merge all identical Note Records??
I found the plug-in that would then let me convert the remaining single-use ones to local notes.
- rob

[tatewise]

Terminology:
Note Records are distinct entities just like Individual records, Source records, Media records, etc.
They have their own tab in the Records Window and in GEDCOM are linked using their Record Id such as @Nxxx@
Local Notes are embedded within records and are attached to such as Facts and Citations and Media, etc.
In GEDCOM they use NOTE, CONT, CONC tags with the note text on the same line.

The primary culprit here is FTM, which has chosen to export its GEDCOM with whatever Note structure it thinks best.
FH is extremely good at importing GEDCOM with minimal changes and keeping as much data as possible, i.e. UDF.
If any data is discarded then that is reported in the Exception Log.

It is quite common for products to create Note Records linked just once instead of using a Local Note.
That is the main reason for the Clean Up Notes plugin, which converts each such Note Record to a Local Note.

If there are lots of Note Records with zero links then either FTM created them or FH V7 has broken the links on import.
Please review the FTM GEDCOM and for each 'zero link' Note Record search for its @Nxxx@ Record Id.
Look for at least one NOTE @Nxxx@ linking to the 0 @Nxxx@ NOTE record that contains the note text.
If you find no NOTE @Nxxx@ links then FTM has produced the 'zero link' Note Records and FH has kept them.
If you find one or more NOTE @Nxxx@ links then FH has discarded them and we must investigate why.
Where are those NOTE @Nxxx@ links located within other records? Post a fragment of GEDCOM here.

The OBJE-NOTE-CONT/CONC structure you mention looks like a Local Note associated with Media OBJEct.
If you perform a similar FH Edit > Find for the note text then does that give a clue to where it is located?
Alternatively, post that fragment of GEDCOM here for me to review.

Finally, merging identical duplicated Note Records is not something that has arisen before, but a Plugin could do it if the manual Edit > Merge/Compare Record is going to be inconvenient.

[rob p]

Mike,
Thanks for your reply. I understand the concept of Note Records and Local Notes, and your explanation of the terminology is excellent.
I also agree that the culprit is FTM’s “problematic” (for lack of a better word) GEDCOM export, and/or the internal workings of the program. –More on that in a minute…

I have far too many to check them all individually, but I’ve checked enough of them (in both the FTM & FH GEDCOMs) to confidently say that none of the zero-count @Nxxx@ NOTE records are linked (no NOTE @Nxxx@ links elsewhere). And the actual text of those said notes does also appear as a Local Note, and where I would expect it (subordinate to the record of interest using NOTE, CONT, CONC tags). FTM did create them.

What was confusing to me was that this happened with only some (of the 1,000’s) of notes in my file. Some (if not most) of my notes came across cleanly as Local Notes, with no Note Record (zero-count or otherwise).

In trying to figure out the multiple single-link (identical) Note Records, I may have stumbled on to a ‘clue’:
In FTM I had used their copy/paste functions for all sorts of things – sources, citations, notes, facts (together with its associated citations, media and notes) and more. I would paste a “copy” and often then modify that new item for its new use (usually a minor edit, like un-linking a particular citation, or changing the word “brother” in a note to “sister”).
Sometimes that “copy” was not altered (for example, copying a fact from one person to another, and all of the associated information is the same). All of the “duplicate” single-use Record Notes i checked were in this category (copied and not altered).
I don’t really want to dig into the FTM exported GEDCOM (1-1/2 million lines!) to see exactly how they handled all of this, and I certainly don’t understand their database, but I suspect that all of my resultant Note Records (zero-linked and single-linked) were a result of this phenomena. I suspect that the zero-linked and the unique single-linked Record Notes all had something to do with FTM’s “copy” functions, and me then altering the copy.

The concept of “Note Records” is new to me. FTM doesn’t present it in the user interface (but apparently does use it ‘behind the scenes’ in some odd manner, related to the “copy” function).
I like it, and I may well use it.
I have far too many duplicate identical Note Records to manually Edit > Merge/Compare Record. How much trouble would a plug-in be? Don’t do it just for me, but I may not be alone here.
- I could just use the Clean Up Notes plug-in and convert everything to Local Notes. (Or just leave it alone.) And I now assume I can delete all of those zero-linked records.

- rob

[tatewise]

There may not be a direct relationship between the internal FTM database and the exported GEDCOM.
Many products have to do their best to translate internal data structures into GEDCOM compliant files and often fail!

It does seem that your copying strategy may be the root cause of some of the unusual Notes.

I am not really in a position to judge whether your zero linked Note records can be deleted. That is a value judgement for you.

At the moment I don't have time to devote to writing a Plugin to merge identical Note records.
It is not a complex process, so why not use it as a way of learning about Plugins?