During a GEDCOM import FH identifies and deals with a number of attributes such as TITL FROM TO ORG REF by treating them as Uncategorized Data.
The data from these can be viewed by using the ALL Tab in the Property Dialogue and is present and correct. However I can find no way of displaying any of this type of data in a Report.
The problem appears to be that they can only be added to the Event/Attribute List as Fact Set 'Custom'. Whereas the addition to a Report requires them to be 'Standard'. I can find no way of changing them from 'Custom' to 'Standard'.
I did try using a WP to change the original GEDCOM file tags to selected Standard 5.5 Tags like:
DATA MEDI NOTE REFN
It worked with NOTE but the other three tags resulted in them being treated as Uncategorized Data. Which surprised me.
Suggestions on how to overcome this problem, please. It effects some 25,000 records!
A rather frustrated Don Bidgood
ID:2817
* Uncategorized Data
-
Jane
- Site Admin
- Posts: 8442
- Joined: 01 Nov 2002 15:00
- Family Historian: V7
- Location: Somerset, England
- Contact:
Uncategorized Data
Those are not valid gedcom tags, you need to work out what the data is and modify it to proper gedcom tags. Many family history programs output invalid gedcom data so you will need to move the data into valid, you also need to ensure the structure of the information is also correct and that the line levels are correct, my guess this is where you went wrong with your search and replace.
Try posting some of the gedcom lines with and I will see if I can spot whats wrong.
Try posting some of the gedcom lines with and I will see if I can spot whats wrong.
Uncategorized Data
I would recomend downloading the Gedcom spec from the downloads section, or jump to http://www.fhug.org.uk/cgi-bin/index.cg ... edCom&id=2.
As a start TITL is not used for names and should be subordinate to either SOUR, OBJE. If a person's title it should be subordinate to EVEN, so should appear something like
2 TITL Lord
FROM and TO are related to dates. Instead of appearing on seperate lines they should appear one line.
E.g. 2 DATE FROM 1837 TO 1866
ORG and REF I haven't come across, though you should be able to work out their context from their values.
If in doubt, changing a UDF to a NOTE should work, but notes aren't valid everywhere. See the standard for details.
The problem with Gedcom is that the standard is not well laid out, there is some ambiguity about what tags mean what, and it can't cope with every program's data structure. And it hasn't been updated for over a decade. Some do their best but use extensions, whilst others follow the spirit but don't follow the spec that closely. Since no program exports a valid 100% Gedcom (not even FH; it has problems with HEAD and OBJE records) trying to ensure strict compliance with the standard is going to be hard work and cause loss of data. The best way would be to handle how the major genealogy programs export their gedcoms and parse them intelligently. E.g. FTM has a problem with putting occupations in PLACE, and has extensions for some events rather than using 1 EVEN/2 TYPE.
As a start TITL is not used for names and should be subordinate to either SOUR, OBJE. If a person's title it should be subordinate to EVEN, so should appear something like
2 TITL Lord
FROM and TO are related to dates. Instead of appearing on seperate lines they should appear one line.
E.g. 2 DATE FROM 1837 TO 1866
ORG and REF I haven't come across, though you should be able to work out their context from their values.
If in doubt, changing a UDF to a NOTE should work, but notes aren't valid everywhere. See the standard for details.
The problem with Gedcom is that the standard is not well laid out, there is some ambiguity about what tags mean what, and it can't cope with every program's data structure. And it hasn't been updated for over a decade. Some do their best but use extensions, whilst others follow the spirit but don't follow the spec that closely. Since no program exports a valid 100% Gedcom (not even FH; it has problems with HEAD and OBJE records) trying to ensure strict compliance with the standard is going to be hard work and cause loss of data. The best way would be to handle how the major genealogy programs export their gedcoms and parse them intelligently. E.g. FTM has a problem with putting occupations in PLACE, and has extensions for some events rather than using 1 EVEN/2 TYPE.
Uncategorized Data
As requested here are a couple of examples:
1 OCCU
2 PLAC World War 1, France
2 TITL Royal Army Medical Corp - Stretcher Bearer
2 REF M821
2 FROM 1914
2 TO 1918
2 ORG Family Photo - J.Southart (Glenister)
In this case all the attributes: OCCU PLAC TITL REF FROM TO ORG are treated as uncategorized data.
1 RESI
2 FROM BEF 1901
2 LOCA Moss Side
2 TOWN Manchester
2 CO Lancashire
2 COUN England
2 REF M897
In this case the attributes FROM LOCA TOWN CO COUN REF are treated as uncategorized data.
In an Individual Narrative Report for this individual there does appear 'He was resident {phone number}.' for each RESI occurrence in the GEDCOM, but nothing else. So the event RESI appears to be acceptable but none of the attributes associated with it are shown.
OCCU is ignored completely. Could this be because FH sees it as an attribute rather than an event?
On the question of valid GEDCOM tags, the list for GEDCOM 5.5 Standard available at http://genealogy.about.com/library/weekly/aa10100d.htm
shows OCCU as a valid tag.
If FH is not using this standard list, can you give me a reference to the list that FH uses?
Thank you both for your responses and interest.
Don.
1 OCCU
2 PLAC World War 1, France
2 TITL Royal Army Medical Corp - Stretcher Bearer
2 REF M821
2 FROM 1914
2 TO 1918
2 ORG Family Photo - J.Southart (Glenister)
In this case all the attributes: OCCU PLAC TITL REF FROM TO ORG are treated as uncategorized data.
1 RESI
2 FROM BEF 1901
2 LOCA Moss Side
2 TOWN Manchester
2 CO Lancashire
2 COUN England
2 REF M897
In this case the attributes FROM LOCA TOWN CO COUN REF are treated as uncategorized data.
In an Individual Narrative Report for this individual there does appear 'He was resident {phone number}.' for each RESI occurrence in the GEDCOM, but nothing else. So the event RESI appears to be acceptable but none of the attributes associated with it are shown.
OCCU is ignored completely. Could this be because FH sees it as an attribute rather than an event?
On the question of valid GEDCOM tags, the list for GEDCOM 5.5 Standard available at http://genealogy.about.com/library/weekly/aa10100d.htm
shows OCCU as a valid tag.
If FH is not using this standard list, can you give me a reference to the list that FH uses?
Thank you both for your responses and interest.
Don.
-
nsw
Uncategorized Data
The point is that Family Historian is just about the best application at sticking to the GEDCOM 5.5 standard. If your file followed the standard then Family Historian would be able to read it, the fault lies with wherever your file came from in the first place.
No one has suggested OCCU is invalid and Family Historian uses it and deals with it correctly. The problem in the case of the example you have given the occupation of the person should follow the OCCU tag which I think is why Family Historian is not bothering to import OCCU at all: its a valid tag but not being used correctly. So rather than:
1 OCCU
2 PLAC World War 1, France
2 TITL Royal Army Medical Corp - Stretcher Bearer
it should be:
1 OCCU Royal Army Medical Corp - Stretcher Bearer
2 PLAC World War 1, France
(although personally I wouldn't class 'World War 1' as a place).
The FROM and TO are not correct GEDCOM:
2 FROM 1914
2 TO 1918
and in correct GEDCOM would be:
2 DATE FROM 1914 TO 1918
REF isn't a valid tag and so
2 REF M821
should probably be put in a note (though would be better ultimately in a source or source citation):
2 NOTE M821
I'm not sure what the ORG tag is being used for but again isn't valid GEDCOM. I suppose it could be added to the note.
Similar issues with the next example you gave. RESI is fine but all the other lines below it are not valid GEDCOM tags so Family Historian can't be blamed for not being able to understand them.
It looks like it will be very difficult to import this data unless you can get it converted into correct GEDCOM and it doesn't look like this is going to be easy to do using Search and Replace. Which program produced this? Maybe someone who uses that program has developed software to convert it into true GEDCOM, it might be worth searching the Internet.
No one has suggested OCCU is invalid and Family Historian uses it and deals with it correctly. The problem in the case of the example you have given the occupation of the person should follow the OCCU tag which I think is why Family Historian is not bothering to import OCCU at all: its a valid tag but not being used correctly. So rather than:
1 OCCU
2 PLAC World War 1, France
2 TITL Royal Army Medical Corp - Stretcher Bearer
it should be:
1 OCCU Royal Army Medical Corp - Stretcher Bearer
2 PLAC World War 1, France
(although personally I wouldn't class 'World War 1' as a place).
The FROM and TO are not correct GEDCOM:
2 FROM 1914
2 TO 1918
and in correct GEDCOM would be:
2 DATE FROM 1914 TO 1918
REF isn't a valid tag and so
2 REF M821
should probably be put in a note (though would be better ultimately in a source or source citation):
2 NOTE M821
I'm not sure what the ORG tag is being used for but again isn't valid GEDCOM. I suppose it could be added to the note.
Similar issues with the next example you gave. RESI is fine but all the other lines below it are not valid GEDCOM tags so Family Historian can't be blamed for not being able to understand them.
It looks like it will be very difficult to import this data unless you can get it converted into correct GEDCOM and it doesn't look like this is going to be easy to do using Search and Replace. Which program produced this? Maybe someone who uses that program has developed software to convert it into true GEDCOM, it might be worth searching the Internet.
Uncategorized Data
It looks to me like the file is from Pedigree and possibly using Gedcom v4. You might be better off asking your source to get the latest version of Pedigree and re-exporting their data in v5.5 format.
The Pedigree User Group seems to have a utility (Ped-FHS v2.6) to convert Gedcom v4 files to v5.5. See here - http://ourworld.compuserve.com/homepage ... isting.htm
The Pedigree User Group seems to have a utility (Ped-FHS v2.6) to convert Gedcom v4 files to v5.5. See here - http://ourworld.compuserve.com/homepage ... isting.htm
-
nsw
Uncategorized Data
Not sure it is GEDCOM 4 but it does look like Pedigree so well spotted John. This has been discussed a few years ago on this forum:
http://www.fhug.org.uk/cgi-bin/index.cg ... op=display
http://www.fhug.org.uk/cgi-bin/index.cg ... op=display
Uncategorized Data
Hi Nick and Jon,
Good detective work on your part. Yes the GEDCOM originated in PediTree and is in the older GEDCOM 4 format. It is the main file for my One Name Study of BIDGOOD and variants and runs to some 30,000 individuals. So that converting it to GEDCOM 5.5 would only be practical if there is software available.
I will take a look at the Pedigree User Group utility you mention Jon, and also the discussion in the FHUG archives that you give Nick, to see if either addresses this problem.
If you have any other suggestions that may help, please post.
I am impressed by FH and the many features that it offers. But I do need to be able to generate Descendant Outline Reports which include all the available information for the individuals including the Uncategorized Data to respond to many of the queries that I receive. Particularly for new researchers GEDCOM is something of a black art!
Thank you for your prompt and helpful response,
Don.
Good detective work on your part. Yes the GEDCOM originated in PediTree and is in the older GEDCOM 4 format. It is the main file for my One Name Study of BIDGOOD and variants and runs to some 30,000 individuals. So that converting it to GEDCOM 5.5 would only be practical if there is software available.
I will take a look at the Pedigree User Group utility you mention Jon, and also the discussion in the FHUG archives that you give Nick, to see if either addresses this problem.
If you have any other suggestions that may help, please post.
I am impressed by FH and the many features that it offers. But I do need to be able to generate Descendant Outline Reports which include all the available information for the individuals including the Uncategorized Data to respond to many of the queries that I receive. Particularly for new researchers GEDCOM is something of a black art!
Thank you for your prompt and helpful response,
Don.
-
Jane
- Site Admin
- Posts: 8442
- Joined: 01 Nov 2002 15:00
- Family Historian: V7
- Location: Somerset, England
- Contact:
Uncategorized Data
If you know anyone with a Mac and Reunion, I saw a post which suggests reunion can import GEDCOM 4 and export 5.5
If might also be worth loading the File in to PAF and exporting it from there to see if that can convert the file up.
Another temporary fix for a copy of your file is to search and replace all the invalid tags with NOTE which should mean all the data shows abet in a silly format on the reports.
If might also be worth loading the File in to PAF and exporting it from there to see if that can convert the file up.
Another temporary fix for a copy of your file is to search and replace all the invalid tags with NOTE which should mean all the data shows abet in a silly format on the reports.