* From TNG > FH5 > TNG

[kbella]

Thanks to you both for your help.

Question: should I apply this fix to the gedcom Colin sent me or a new one generated from my TNG website?

[tatewise]

I will leave Colin to answer that question, because I don't know who sent what to whom.

The PAGE (Where within Source) problem appears to be caused by multiple lines that TNG allows to be entered into that field.
When TNG exports to GEDCOM it does not handle those multiple lines correctly, because GEDCOM does not allow them.
Kathleen, can you confirm whether that is a plausible explanation or not?

[Valkrider]

Kathleen

Create a new TNG Gedcom and apply Mike's plugin. I manually edited the file and tried an import into a new project and it worked fine. Even though PAGE doesn't support CONC it seems to have imported the data as a 4th level element. Mike's way is the better way or even better still is to not create the problem in the first place when you ender data into TNG.

TNG should not allow you to create a PAGE entry with more than 248 characters and so this is a bug in TNG by the look of it.

[kbella]

Mike and Colin, I have emailed Darrin Lythgoe and asked him to read this thread. I have no idea what PAGE is or how to fix it. Perhaps he can tell me what I need to fix in TNG to make this go more smoothly. I'm wondering if it is something I have copy/pasted into note fields or something similar? I can deal with manual edits if I know what to fix.

Thanks again, I will let you know when/if I hear from Darrin.

[tatewise]

PAGE is a Source Citation field (not a Note field) that in FH and GEDCOM 5.5 is called Where within Source.
GEDCOM 5.5 defines it as:

Specific location within the information referenced. For a published work, this could include the volume of a multi-volume work and the page number(s). For a periodical, it could include volume, issue, and page numbers. For a newspaper, it could include a column number and page number. For an unpublished source, this could be a sheet number, page number, frame number, etc. A census record might have a line number or dwelling and family numbers in addition to the page number.

If Kathleen could search for the EXCLUDED: invalid lines such as: "Chivers Julia, Clutton 11 96" or "Surname: McElrea" as reported above, then that will confirm the location. The Individual Gedcom Id=I96 and Individual Gedcom Id=I180 should help identify which Individual Records are involved.

[Valkrider]

Mike

Looking at her Gedcom all of those excluded ones are excluded from the second line of a PAGE entry in the Gedcom.

I suspect looking at her Gedcom she is using PAGE incorrectly and should be using NOTE instead but not knowing her workflow I cannot be certain. However, TNG should be handling this correctly IMHO and throwing an exception when creating the Gedcom rather than creating an invalid one.

[kbella]

Mike,

Can you give me a short explanation of what seems to be wrong that I can send to Darrin Lythgoe?

Thanks,

[tatewise]

The PAGE (Where Within Source) problem appears to be caused by multiple lines that TNG allows to be entered into that field. When TNG exports to GEDCOM it does not handle those multiple lines correctly, because GEDCOM 5.5 does not allow them. It results in invalid GEDCOM code such as:

2 SOUR @S18@
3 PAGE 1st line of text
2nd line of text
3rd line of text
3 DATA

which should be:

2 SOUR @S18@
3 PAGE 1st line of text 2nd line of text 3rd line of text
3 DATA

Or even better, the multiple lines should not be allowed into that PAGE field initially.

[tatewise]

I have made the Plugin a bit safer to use, with progress dialogues, and exit option.

Just single-click it below to install over the previous version.

[kbella]

Thanks, Mike. Darrin sent me a couple of files to install on my TNG site. I will try and get to that today and see what happens.

Kathleen

[kbella]

Mike,

Finally got my TNG installation updated, installed the 2 files Darrin sent me and exported a gedcom. Attached is the report I got from FH. The validation report showed no errors. I ran the plug-in you sent me and it fixed "0" problems.

[Valkrider]

Kathleen

Read the bit at the top of the report

Code: Select all

Lines marked "EXCLUDED:" indicate data which has not
  been loaded, and explain why it wasn't loaded.

  Indented lines marked "EXCLUDED BRANCH LINE:" indicate
  data lines which are not necessarily invalid in
  themselves but which have had to be excluded as a
  consequence of excluding other lines.

  Lines marked "INFO ONLY:" indicate data that has
  been successfully loaded, but where special action
  was needed to load it (e.g. GEDCOM errors detected
  and automatically corrected by Family Historian).

The vast majority are INFO ONLY type of errors.

All the map data is EXCLUDED

I only spotted one other EXCLUDED report on a quick scan and that is where you have a persons sex as U. You can just manually edit that record.

[tatewise]

The Exception Report identifies a number of issues, some of which should be investigated by Darrin Lythgoe.

(1) The PAGE Problem (Refer to Darrin Lythgoe)
This has the main focus of earlier discussions and is not fixed correctly.
TNG exported GEDCOM for multiple PAGE lines now uses the CONTinuation tag which is NOT valid for PAGE tags.
The PAGE tag does NOT support multiple lines, and by inference neither CONT tag nor CONC tag.
I thought my posting of Thu Jul 03, 2014 6:56 pm made that clear.

(2) EXCLUDED 1 SEX U Invalid (Refer to Darrin Lythgoe)
The SEX tag does NOT allow the U value, only M or F.
Nevertheless, in FH the three Individuals with Id=3402, 3449 & 3537 will have their SEX undefined OK.

(3) INFO ONLY: Tags with Invalid Values (Refer to Darrin Lythgoe)
Several Tags namely 1 IMMI, 1 BAPM, 1 RESI, 1 CENS have values that are NOT allowed in GEDCOM.
In Exclusion Report reported as INFO ONLY: Detected & fixed field format error (data moved to Note Field).
Some have multiple CONTinuation lines that are also not allowed in GEDCOM for these Tags.
FH has moved the values to the associated NOTE field, with additional UDF for the CONT lines.

(4) INFO ONLY: Loaded uncategorised data
These mostly custom Tags can be handled later as UDF.
See how_to:handling_unrecognised_data_fields|> Handling Unrecognised Data Fields.

(5) EXCLUDED 0 _PLAC Map Lat/Longitude Records
These are custom GEDCOM Records that are NOT recognised by FH and excluded.
They form the vast majority of the entries in the Exception Report.
They would need a Plugin to convert to some standard GEDCOM format associated with Place names.

[kbella]

Thanks Mike and Colin,

Yes, I read the statement at the top of the file and realize that a lot (probably most) of the report has to do with mapping. I was having a hard time finding the data that had been moved to note fields, but I'm getting better at it.

Thank you, Mike, for that summary to send to Darrin.

Thanks for all your help. I will report back with any news, as I suspect I won't be the only person to ever do this and at least the steps will be here for the next victim.

[tatewise]

Please refer to how_to:handling_unrecognised_data_fields|> Handling Unrecognised Data Fields for advice on finding and fixing UDF.

I have created an initial draft of how_to:import_from_tng|> Import from The Next Generation (TNG) that can be updated as things progress.