Handling Uncategorised Data Fields (UDF)
Uncategorised Data Fields (UDF) are data fields that Family Historian (fh) does not recognise as its own GEDCOM Data References, but have been retained for review. These are often fields that are user-defined by other products, such as _UID (User Identifier) and _AKA (Also Know As) in PAF, or _MILT (Military) and _FOOT (Footnote) in Ancestry and FTM, or completely invalid GEDCOM fields such as CE81 in Generations.
There are too many products and too many UDF to list them all, so some investigation is needed to establish what data values they hold, and whether that is important. If necessary ask for advice in the Forums.
So a UDF will have a Data Reference Tag such as _UID and a UDF Item value such as 123ABC. When a UDF is deleted, both the Data Reference Tag and UDF Item value will be deleted.
The first time a GEDCOM file is opened by fh, (including opening a file from a previous Version of fh), it will report all UDF and other issues in the Exception Report, but once Saved they will no longer be reported on load. The report can be viewed using Window > Log Files.
With fh V6.1.4, Smart GEDCOM Loading was introduced, where some popular user-defined Data Reference Tags are automatically converted to fh compatible tags and reported in a Process imported Gedcom extensions log.
Each Exception Report for a UDF typically has the format:
I.6789 - INFO ONLY: Loaded uncategorised data (non-GEDCOM)
above this will be a line with the format:
Record Type=Individual. Gedcom Id=P4321. Record Number=1234.
where the Record Type may also be Family, Source, etc; the Gedcom Id= gives the pre-import record id; and Record Number gives the post-import record id.
If you are happy there is no useful data mentioned in the Exception Report, then the quickest way to proceed is to open Tools > Preferences > File Load/Save and select the two exclude options. The next time the Project is opened, all UDF will be deleted, but you must use Save to commit the changes to the GEDCOM file. We advise restoring the two options to their Installation Settings.
By default fh will load all UDF fields, and they will appear in the Property Box on the All tab with a * star bullet point next to them. Use the Tools > Uncategorised Data > Find Uncategorised Data command to obtain a Named List of affected records.
Since fh V5 the easiest way to find all UDF is to the use the UDF List Plugin. It will list full UDF details, and you can selectively:
- Convert some UDF using a custom Plugin for the other product (see Import from... advice for that product)
- Convert some UDF to valid fields using Tools > Uncategorised Data > Recover Uncategorised Data
- Convert some UDF to fh Standard/Custom Facts using the Change Any Fact Tag Plugin (see below)
- Manually copy the values to valid fields, and delete the UDF
- Delete the UDF if of no interest
To delete UDF, select the desired fields and press the keyboard Delete key. In the Result Set of the UDF List Plugin, select any from the first column UDF Item, and Delete. There will be a warning message before confirming with OK. If after checking the effects, you change your mind, use Edit > Undo Delete Selected Data Items BEFORE closing fh.
Alternatively, it may be possible to change settings in the other product and export the GEDCOM file again.
Change Any Fact Tag Plugin
Many other products allow Facts to be added that have user-defined GEDCOM fields, which fh treats as UDF, but can be converted to conventional Standard/Custom Facts by following the steps below.
- The first task is to identify those candidate UDF field Tag names.
- One method is to run the UDF List Plugin, and in its Result Set examine the Item Data Reference column. All the candidates will only have two Tag names separated by a dot such as INDI._ELEC or INDI._MILT or FAM._SEPR, where the first tag is either INDI or FAM and the second tag often begins with an underscore (_). Click on the Item Tag or Item Data Reference column heading to sort into order.
- Another method is to run the Change Any Fact Tag Plugin, set the Source Tag Set: to <UDF Tags>, and the Source Tag Name: drop-list will identify all candidate INDI UDF tags. Open the Family Records tab and repeat to identify any candidate FAM UDF tags.
- The second task is to discover each Fact Name used in the other product, so it can be represented in fh. This may involve some detective work, but some popular tags and names are listed later. Where possible, consider using GEDCOM Standard Facts, or existing Fact Sets such as the Extended Set, instead of creating new Custom Facts.
- Download & install any required Fact Sets to create sets of Custom Facts.
- To create a brand new Custom Fact use Tools » Fact Types, click the New button, enter the Fact Name, choose the Fact Type & Record Type, then click Create, and in the Fact Definition dialogue click OK (the details can be adjusted later). Mistakes can be corrected by using the Delete button, and starting again with the New button.
- Finally, run the Change Any Fact Tag Plugin, and click the Obtain Help & Advice button for details.
- Set the Source Tag Set: to <UDF Tags>.
- Select a Source Tag Name: from its drop-list.
- Set the Target Tag Set: to Standard for Standard GEDCOM Facts, or the name of a downloaded Fact Set, or Custom for brand new Custom Facts.
- Select the desired Target Tag Name: from its drop-list.
- To perform the change click the green button below and use the Confirm Edit dialogue.
- Repeat the above steps for each UDF, and switch to the Family Records tab as necessary.
Unfortunately, fact UDF Tags will often have subsidiary UDF tags, which the Plugin does its best to correct, but if it cannot they become Source Note text, that will need manual edits or another custom Plugin to fix them.
UDF Tags & Fact Names
Since fh V6.1.4 a new Fact Set called Extended Set is automatically installed that supports Ancestry and FTM predefined facts, so many of the facts listed below are handled automatically.
For each UDF Tag this lists its Fact Name and recommended Fact Type & Fact Record for the Tools » Fact Types » New dialogue. The Possible Alternative may be a Standard Fact, or downloadable Custom Fact in a Fact Set, or another way of handling that UDF Tag.
This list is currently focussed on Ancestry and FTM, but others can be added.
|UDF Tag||Fact Name||Fact Type||Fact Record||Possible Alternative|
|_CIRC||Circumcision||Event||Individual||Standard Physical Description|
|_DCAUSE||Cause of Death||Attribute||Individual||Migrate to Standard Death Event Cause using dedicated Plugin|
|_DEG||Degree||Attribute||Individual||Standard Graduation or Education|
|_FUN||Funeral||Event||Individual||Standard Burial or Cremation|
|_HEIG||Height||Attribute||Individual||Military History Height (Military) or Standard Physical Description|
|_MDCL||Medical Condition||Attribute||Individual||Military History Medical History|
|_MILTID||Military ID||Attribute||Individual||Military History Military Service No.|
|_MILT||Military Service||Attribute||Individual||Military History Military Service|
|_WEIG||Weight||Attribute||Individual||Military History Weight (Military) or Standard Physical Description|
|ADDR||Web Address||Attribute||Individual||Migrate ADDR, EMAIL, PHON data to a Contact Info Custom Attribute|
|Attribute||Individual||See Saving Contact Info for Living People|
|PHON||Phone Number||Attribute||Individual||Then delete these three UDF Tags|