* Gedcom import errors

[TMG_refugee]

I migrated from TMG to Rootsmagic 7. I now see that Family Historian may have many more functions than RM7. My imports from RM7 create many errors. I have tried to isolate the issue and finally created a clean and empty new project in RM7 and imported into FH6. I still have many many errors. They seem to point back to the event in RM7. This is what I see:

l.2409 - EXCLUDED: invalid line : "0 _EVDEF BIRT"
l.2410 - EXCLUDED BRANCH LINE: : "1 TYPE P"
l.2411 - EXCLUDED BRANCH LINE: : "1 TITL Birth"
l.2412 - EXCLUDED BRANCH LINE: : "1 ABBR Birth"
l.2413 - EXCLUDED BRANCH LINE: : "1 SENT [person] was born< [Date]>< [PlaceDetails]>< [Place]>."
l.2414 - EXCLUDED BRANCH LINE: : "1 ROLE Witness"
l.2415 - EXCLUDED BRANCH LINE: : "2 SENT [ThisPerson] witnessed the birth of [person]< [Date]>< [PlaceDetails]>< ["
l.2416 - EXCLUDED BRANCH LINE: : "3 CONC Place]>."
l.2417 - EXCLUDED BRANCH LINE: : "1 ROLE Doctor"
l.2418 - EXCLUDED BRANCH LINE: : "2 SENT [ThisPerson] delivered [person]< [Date]>< [PlaceDetails]>< [Place]>."
l.2419 - EXCLUDED BRANCH LINE: : "1 PLAC Y"
l.2420 - EXCLUDED BRANCH LINE: : "1 DATE Y"
l.2421 - EXCLUDED BRANCH LINE: : "1 DESC N"

I see similar type errors for Death etc. but not all event in RM7.
I would like to correct this in RM7 before finally importing to FH6. Any suggestions?

I also tried the mapping tool but found many of my addresses ended up in very unrelated places. I went back to RM7 and on a small project geocoded all the places. this seems to work when imported into FH6. I did not see an explanation of what and address must look like for FH6 to map it. Any suggestions Here?

A third question relates to the info only uncategorised import errors. I looked at help but just saw that some fields like dates may not work as dates. Any explanation on how to deal with these errors?

[tatewise]

I have move this to the Importing and Exporting Forum that seems more appropriate.

Firstly, if you still have TMG and a Project that is not to far out-of-date, then the best option may be a direct import from TMG to FH, as most TMG issues have been fixed, and if you wait a short while for FH V6.1 then it fixes a few more. See http://www.family-historian.co.uk/downl ... ade-to-6.1 for details.

See also how_to:import_from_tmg|> Import from The Master Genealogist (TMG) for general details although they need updating for FH V6.1.

I am surprised you are getting so many issues with RM7 because its support of GEDCOM is among the best.

See how_to:import_from_roots_magic|> Import from Roots Magic (RM) and check the Destination box has General selected as from the errors I suspect the export is a RM7 backup dialect of GEDCOM for its own use.

The internal mapping feature in FH is only intended to geolocate Places (towns/cities) not Addresses (houses/streets).
The normal comma separated format for Places & Addresses is fine.
There are Plugins that use Google mapping for both Places and Addresses.

See how_to:index#importing_to_family_historian|> Importing to Family Historian under Use the following advice in the order listed to repair common residual problems. especially how_to:handling_unrecognised_data_fields|> Handling Unrecognised Data Fields.

[TMG_refugee]

Thank you for the response and pointers. TMG was great and now I want to find a permanent replacement. I followed your pointers but did not find an answer to my problem.

I have been in Rootsmagic for over a year and have added many facts, people and even new family branches so going back to TMG would be a very last resort.

Given the error message and looking at the gedcom from a completely empty RM7 project there must be something that is in error at the actual event in RM7. Does FH6 have the capability to tell me exactly what it did not like or just the fact that it did not like it?

If I need to correct from the RM67 side that is ok but I'm at loss as to what to change if I don't know why FH6 is rejecting it.

With what I have found so far FH6 is great but I need to cleanup the old data somehow.

[tatewise]

I have a free version of RM6 and tried a few experiments.

Our how_to:import_from_roots_magic|> Import from Roots Magic (RM) advice needs updating, and relies on migrants such as yourself to notify problems.

The error reports such as "0 _EVDEF ****" are fact definitions for RM7 akin to the FH Tools > Fact Types.
Error reports for "1 _TMPLT " and successive lines, seem to be RM7 display templates akin to FH Property Box > Customize.

These are easily eliminated by using RM7 File > Export and untick Extra details (RM specific) which is good clue!
Also probably wise to untick Note formatting (bold, etc) as FH does not support such formatting.
I will add this advice to the how_to:import_from_roots_magic|> Import from Roots Magic (RM).

That still leaves a few Uncategorised Data Fields (UDF).
e.g.
"1 _UID 259A33F2314D4EC490EC74FA978E0148CCD3" these are RM7 record id that may be deleted.
"2 _TYPE DOCUMENT"
"2 _TYPE PHOTO" these four seem to be Media properties that mean nothing to FH and may be deleted.
"2 _SCBK Y"
"2 _PRIM N"

Other reports such as "1 EVEN Basic Research" will have moved the text Basic Research into a local Note.

Do you get much else beyond the above?

[DavidNewton]

Just a thought. The piece of GEDECOM in the OP looks to me like a custom fact definition.

David

I see I was late saying that.

[TMG_refugee]

I have done further testing using the error report from FH6, the export from RM7 and trying to figure out where thee errors are coming from.

I mapped the standard fact types from FH6 against the fact types in RM7. There are some items that equate but many that don't. Add in the custom facts that were created first in TMG and then in RM7 and I have many errors.

I created an empty project in RM7 and exported it to FH6 - no erors.

I then started adding the facts that were common to both such as birth, death and still no errors.

I then started looking at the gedcom standard 5.5 and 5.51. Among the many fact types there there is one named EVEN for an event that will contain data about something. It looks pretty much a free form type of event.

I entered one of these in RM7 and exported to FH6 and it did not like that tag. It went the route of handling uncategorized data and when outputting a report the wording is not at all reasonable. If the EVEN fact is part of both standards why is FH6 rejecting or editing this event fact?

I got the following error codes when importing the EVEN fact type:
l.31 - INFO ONLY: Detected & fixed invalid use of EVEN (event) tag: "1 EVEN A military driver in company b."
l.36 - INFO ONLY: Detected & fixed invalid use of EVEN (event) tag: "1 EVEN This is a free form description"

I have not studied the gedcom standard to be able to identify the problem for syntax etc. but the export from RM7 appears to follow that 5.5 and 5.51 standard.

Anyone know how to identify just what FH6 doesn't like about that Fact?

Here is the entire gedcom export for my beginning test.

0 HEAD
1 SOUR RootsMagic
2 NAME RootsMagic
2 VERS 7.0.11.0
2 CORP RootsMagic, Inc.
3 ADDR PO Box 495
4 CONT Springville, UT 84663
4 CONT USA
3 PHON 1-800-ROOTSMAGIC
3 WWW www.RootsMagic.com
1 DEST RootsMagic
1 DATE 19 JAN 2016
1 FILE Test for FH 3.ged
1 GEDC
2 VERS 5.5.1
2 FORM LINEAGE-LINKED
1 CHAR UTF-8
0 @I1@ INDI
1 NAME Joesph /Smith/
2 GIVN Joesph
2 SURN Smith
1 _UID 7C5F5271974D48408D74610D7336EF7182D5
1 CHAN
2 DATE 19 JAN 2016
1 NOTE Joe Person Note
1 BIRT
2 DATE 1 JAN 1980
2 PLAC Manchester, Connecticut
2 ADDR 175 Pine St
2 NOTE Note for Joe's Birth
1 EVEN A military driver in company b.
2 TYPE Military
2 DATE 2 JAN 2013
2 PLAC Toul, France
2 NOTE See ?? for his complete service record
1 EVEN This is a free form description
2 TYPE Misc
2 DATE 5 JAN 2014
2 PLAC Glastonbury, Connecticut
2 ADDR Library
2 NOTE joes miscellaneous note
1 OCCU factory worker at Colt Factory
2 DATE 2 JAN 2015
2 PLAC Hartford, Connecticut
2 ADDR 122 Trumbull st.
2 NOTE Joes occupation note
1 DEAT
2 DATE 1 JAN 2016
2 PLAC East Hartford, Connecticut
2 ADDR 25 Main st.
2 NOTE Joes death note
0 TRLR

[AdrianBruce]

The EVEN line should not have anything else after the word EVEN. This is the 5.5 standard on EVEN:

n EVEN {1:1}
+1 <<EVENT_DETAIL>> {0:1} p.29

Note that there is nothing else on the EVEN line (apart from the {1:1} bit, which means - exactly one please). Really, the "A military driver in company b" should be in the Note.

But what I don't know is how to fix that - hopefully someone else will come along with that knowledge.

Adrian

[tatewise]

EVEN in Gedcom defines a custom Event, but they are not allowed a description value in the way that an Attribute such as Occupation allows.

The error report does not mention Unrecognised Data and does not treat the EVEN tag as an error, but its use is invalid and corrects that by moving the descriptive value text to a local Note. If you check the Military and Misc Facts in FH you will find both are OK, no UDF, but the text is in their local Note box.

A common problem with many genealogy products is that they misinterpret the Gedcom 5.5 spec and also add proprietory tags only understood by that product. It is quite normal for import errors to be reported in such cases, but also many products simply discard what they don't like without reports. FH gives comprehensive reports and discards very little, although the wording of some errors could be better.

[DavidNewton]

I believe the problem, as with so many import problems, lies in the fact that the two GEDCOM standards are not the same and as with most file format changes software sticking to the older version will have problems reading the newer. The examples below are taken from the GEDCOM standard documents. The first from the Final Draft of GEDCOM 5.5 dated Jan 2 1996, the second from the Draft Release of Gedcom 5.5.1 dated 2 Oct 1999

Gedcom 5.5
1 EVEN
2 TYPE Election
2 DATE 07 NOV 52
2 NOTE Candidate for State Senate.

Gedcom 5.5.1 example
1 EVEN Appointed Zoning Committee Chairperson
2 TYPE Civic Appointments
2 DATE FROM JAN 1952 TO JAN 1956
2 PLAC Cove, Cache, Utah
2 AGNC Cove City Redevelopment

David

[PeterR]

Unfortunately that DRAFT Release 5.5.1 document contains a few inconsistencies. The example quoted is on page 48 in the PDF version and violates the formal definition for the EVEN tag given on page 34 in the same document; that definition is unchanged from the official Release 5.5.

[DavidNewton]

Precisely, it is a Draft release prepared by the Family History Department of The Church of Jesus Christ of Latter-day Saints, and yet contains an example of an apparently illegal use of a tag which is unchanged from version 5.5.

We can speculate as to why that is - did they make the change in an earlier Draft which has not seen the light of day; perhaps the team writing this example failed to communicate with the team writing the formal draft. In the end the unfortunate thing, which cannot be altered, is that this Draft which was never approved, was published and apparently not checked for errors. The net result is this thread and doubtless hundreds of others wondering why the GEDCOM format is causing us so many problems when we try to transport our data.

David

[tatewise]

In Modifications in Version 5.5.1 on Page 6 of the Draft 5.5.1 in it says very clearly:

Added a generic FACT tag to the individual attribute structure. Previously, the generic EVEN tag was used. The FACT was added to give a semantic difference between generic events and generic facts or characteristics (see <<INDIVIDUAL_ATTRIBUTE_STRUCTURE>>, page 33.)

So the EVEN tag has NOT changed, but FACT has been added for Attributes and the example should read:

1 FACT Appointed Zoning Committee Chairperson
2 TYPE Civic Appointments
2 DATE FROM JAN 1952 TO JAN 1956
2 PLAC Cove, Cache, Utah
2 AGNC Cove City Redevelopment

But it seems many developers have worked from the faulty example rather than the formal specification, and have ignored the very obvious change described on Page 6. I rest my case, m'lord.

[DavidNewton]

But it seems many developers have worked from the faulty example rather than the formal specification, and have ignored the very obvious change described on Page 6. I rest my case, m'lord.

Amen to that. I currently have FH, FTM 2014, RootsMagic 7 and Legacy 8 installed (and if anybody is interested it took me a while to decide which one to use). I constructed a very small .ged with one individual, a birth date of 1/1/1900 and the above FACT and imported it to each one in turn. None of them recognised the FACT tag as valid. RM7 and FTM 2014 stated the tag as unknown and discarded it; FH imported the tag and it's subordinates as UDFs but threw nothing away; Legacy 8 recognised the tag as unknown before importing and offered to create an event for it. I chose yes to that and named it test. After import the name test had been discarded and the TYPE value used as a name. On export Legacy reproduced the form of the bad exmple in the GEDCOM 5.5.1 draft standard, FH exported it basically unchanged and both FTM 2014 and RM7 had erased all trace.

As an extra both RM7 and L8 added a DEAT tag with value Y, although Legacy did ask if the individual was still living.

David

[tatewise]

DEATH or any other Event with value Y is perfectly valid GEDCOM.
It is used to signify that an otherwise empty Event must be retained.
Checkout the spec 5.5.

[TMG_refugee]

Thank you for all the info on Gedcom standards and how everyone is treating them, software wise. I am trying for a solution and have tried the following with a small one person project.

1. Export from RM 7 without the extra details (rm specific).
2. I edit the gedcom file created in step 1 and change all EVEN tags to _ATTR tags.
3. In FH6 I created tags for my recognized fact types with a type of attribute.
4. Import into FH6. No errors and it appears that the sentences are being rendered correctly in FH6 output.

Is this going to a reasonable, albeit an manual process, and will it cause me more issues later on?

This approach would be labor intensive and I would not expect some of the people I collaborate with to do this. Other solutions?

[tatewise]

That sounds like it would work, but a solution is possible using a Plugin to automate it all.

Furthermore, if you use the Export Gedcom File Plugin for say RM7 it converts _ATTR with value back into EVEN with value on same line. So you get back to where you started.

[AdrianBruce]

.... the unfortunate thing, which cannot be altered, is that this [GEDCOM 5.5.1] Draft which was never approved, was published and apparently not checked for errors. ...

I believe that it was never published. That's simply a version that was out for consultation and someone decided to promulgate it. In fact, I've always had the sneaking suspicion that I've seen another 5.5.1 document. From probably fallible memory, the presentation of the document was different, so I've no idea if the contents were also different.

What's even more absurd is that the 5.5.1 individual event structure doesn't have the EVENT_DESCRIPTOR used as an example, but the family event structure does!!!!

Thank you Mike for proving my cynical view that 5.5.1 has more holes in it than a Gruyere....

[DavidNewton]

Perhaps "published" is the wrong word. Nevertheless, the online document is freely available from FamilySearch

https://familysearch.org/developers/docs/gedcom/

David

[tatewise]

Yes, but what does it say on the very first page?

This document may be copied for purposes of review only. It must not be used for programming of genealogical software while in draft.

I rest my case again m'lord.

[AdrianBruce]

... the online document is freely available from FamilySearch ...

Which is even weirder - because originally you had to seriously hunt for that across the Internet - it was absolutely not on an LDS site! I got my copy off a site that had nothing whatever to do with them.

[TMG_refugee]

Just as an FYI

When I import from an RM7 export, the sources do not import correctly. The first is what gets exported from RM7. When imported into FH6 all is not well as you can easily determine. If I correct the source in FH6 and then export from FH6 I get the second description. It appears that the RM7 export is in error. The second description seems to be more accurate. Both data entry screens for RM7 and FH6 were filled out identically for title, author etc.

0 @S1@ SOUR
1 ABBR History of Manchester Connecticut
1 TITL Percy W. Bidwell & Mathias Spiess, <i>Manchester Official Souvenir Prog
2 CONC ram</i> (Manchester, Connecticut: Town of Manchester, 1924).
1 TEXT Also <i>Manchester Official Souvenir Program, Welcome Home Day,Saturday
2 CONC , May 17, 1919</i>

0 @S1@ SOUR
1 AUTH Percy W. Bidwell & Mathias Spiess
1 TITL Manchester Official Souvenir Program, Welcome Home Day,Saturday, May 17, 1919
1 ABBR History of Manchester Connecticut
1 PUBL Manchester, Connecticut: Town of Manchester, 1924.
1 CHAN
2 DATE 22 JAN 2016
3 TIME 12:34:48

[tatewise]

Yes, I am continuously amazed at how professional software developers misinterpret a comparatively simple specification.
Do they do it deliberately to make it difficult to migrate to other products?

The FH interpretation is of course perfect, which is one of its USP.

The CONC lines are perfectly OK, although a little premature, because the text could all go on one line easily.

A Plugin could be written to juggle the fields into a more probably correct arrangement.

[AdrianBruce]

The RM interpretation seems to be the result of cramming a citation (comprising Author, Publication and Title) into the GEDCOM Title field.

Do you know if RM was using a citation template to enter that data? If so, the Author, Publication and Title may be held separately in RM but as elements of a templated entry, rather than as the native Author, Publication and Title? Just guessing.... Which might make a corrective plug-in difficult if someone else is using a different template.

[DavidNewton]

RootsMagic has numerous templates but when importing sources I believe that generally the template information is converted into a non-templated "Free-form" source. Creating a new "Free-form" source in RootsMagic offers the following fields: Master Source, Footnote, Short footnote, Bibliography on the first tab; Source Reference Number, Source Text, Source Comments on the second tab and two further tabs to attach media and set a repository. Using RM7 I created a new source with the name "Freeform text source" and filled in the fields on the first two tabs with "This is the <fieldname>". On exporting to gedcom without rm specific extras gives

0 @S15@ SOUR
1 ABBR FreeForm test source
1 REFN This is the Source Reference Number
1 TITL This is the footnote
1 TEXT This is the Source text
1 NOTE This is the Source Comments

As can be seen the Short footnote and Bibliography are removed. Adding in the rm specific items gives significantly more

0 @S15@ SOUR
1 ABBR FreeForm test source
1 REFN Source Reference Number
1 TITL This is the footnote
1 _SUBQ This is the Short Footnote
1 _BIBL This is the Bibliography
1 TEXT This is the source text
1 NOTE This is the Source Comments
1 _TMPLT
2 TID 0
2 FIELD
3 NAME Footnote
3 VALUE This is the footnote
2 FIELD
3 NAME ShortFootnote
3 VALUE This is the Short Footnote
2 FIELD
3 NAME Bibliography
3 VALUE This is the Bibliography

Whether or not this is helpful I don't know.

David

[tatewise]

Referring back to the TMG_refugee posting on Fri Jan 22, that example of RM7 exported Source record is different again, so presumably is using a different RM7 Template?