* FH GEDCOM 5.5.1 compliance

[KFN]

In Places and Addresses (20441) wildbill said:

. My disappointment with the FH Address is that it is not relational to the Place

In GEDCOM the ADDR (address tag) is very separate from the PLAC tag and are not related in any way except that they are subtags of an event or attribute (facts).

The ADDR tag also inconsistent in its current (v5.5.1) design in that it is has two distinct places to enter address data, one that does not have separate tags for address parts and one that does! HOWEVER the GEDCOM documentation clearly states not to use the separate address parts in its statement:

The address structure should be formed as it would appear on a mailing label using the ADDR and the CONT lines to form the address structure. The ADDR and CONT lines are required for any address. The additional subordinate address tags such as STAE and CTRY are provided to be used by systems that have structured their addresses for indexing and sorting. For backward compatibility these lines are not to be used in lieu of the required ADDR.and CONT line structure.

If I recall correctly FTM, RM7 and some other genealogy programs do not support the ADDR tags as subtags to an event/attribute (facts) anyway, they make you enter them as a “Fact”.

If FH supports GEDCOM correctly breaking out a street address (without including City, state, and other address information) to add to any PLAC tag for location use on a map would add some uncertainty about what to grab to the code and therefore in my mind would indicate that street address should be included (therefore cemetery, school names, grave plots, etc) as part of the PLAC tag if you want to map this detail specifically.

[tatewise]

GEDCOM is only meant to be a GEnealogy Data COMmunications format and does not restrict internal program data structures.
It only advises not to use separate address parts for backwards compatibility within the GEDCOM file. Again that imposes no constraint on internal program data structures.

The PLAC and ADDR fields are associated via their parent Fact. It is FH that chooses not to associate them in the Tools > Work with Data lists and elsewhere such as in the Map Window.

FH has chosen to use GEDCOM as its external database format, which is a good idea.
However, it is very clear that FH has an extensive internal database structure that is far more complex.
So there is little to prevent ADDR being subsidiary to PLAC if desired.

[KFN]

Tatewise,

GEDCOM is only meant to be a GEnealogy Data COMmunications format and does not restrict internal program data structures.

Yes this is true, but my understand was that FH7 used GEDCOM internally as well! Is this not true?

[tatewise]

Depends on what you mean by "use GEDCOM internally as well".
Yes, it has records matching each of the GEDCOM records, and fields within those records that mimic the GEDCOM structure, so that Data References such as INDI.BIRT.PLAC refer to a GEDCOM field.
However, FH does not manipulate a GEDCOM text file internally.
It maps each data element to an internal database structure in local memory and supplements it with additional details to make data access more efficient and provide extra features such as Relational Pool numbers to name just one.

[KFN]

I just start to investigate FH7 to use in conjunction with my very GEDCOM 5.5.1 compliant database. I probably use every (or nearly every) tag in the GEDCOM specification except those specific to LDS. Based on your statement I’m probably going to find some lost data or broken concepts at some point in my investigation. That would be disheartening!

[tatewise]

I've split this off as a separate topic because it is straying away from the OP.

I don't understand why you think you are going to find some lost data or broken concepts.
FH maps every GEDCOM file record and field into its internal database records and fields.
It reads in GEDCOM 5.5.1 files and saves/exports GEDCOM 5.5.1 files.
So theoretically the output file should be logically identical to the input file if no changes have been made.

However, its interpretation and support of some fields differs from some other products, although IMO FH is among the best.
e.g. The fact TYPE field is not well supported as discussed in Question: Marriage Status (17530).

Are you aware of John Cardinal's https://www.gedcomassessment.com/ tables and utilities?
However, only FH V6 is listed although I have sent an assessment for FH V7 which is much better.

[KFN]

Yes I’ve seen and reviewed a few of the tests from John’s site, but some time ago!

I don’t recall my findings in specifics, but I do remember him not testing things like PLAC.FORM or some other PLAC subtags.

FACT.TYPE was a concern with all programs, which for me is a deal breaker!

My point was with his tests is that many were not as comprehensive as I would have liked! They were on the surface very good but not looking very deep at v5.5.1

Thanks for breaking this out as you are correct that I took the OP entry off topic.

[AdrianBruce]

... Based on your statement I’m probably going to find some lost data or broken concepts at some point in my investigation. ...

There is no logical reason why mapping in both directions between the GEDCOM 5.5.1 file and FH's internal tables can't be 100% accurate. Certainly any code written to conduct a mapping may contain errors of design or coding, but I think that facing up to reading and writing GEDCOM 5.5.1 every time that data is read from or written to those internal table, is far more likely to push the designers and coders of FH to think closely and accurately about that mapping. Other software, however, can regard the GEDCOM mapping to / from internal files as a spray-on extra that many of their customers will never use (witness the number of "What is GEDCOM?" articles I see) and in some cases their designers seem not to have understood the format anyway.

If you were wondering if GEDCOM could be used internally all the time in FH, the answer would appear to be, effectively, no. There's a guy named Tamura Jones who is a commentator and scourge of sub-standard genealogy software. One article in his blog, IIRC, mentions that several programs have tried using GEDCOM both as a file format and for internal manipulation. Tamura's verdict on them was that they were poor (slow???) - whereas, he said, FH did it right, by using internal non-GEDCOM tables. Believe me, a compliment from Tamura is to be treasured!

This isn't to claim that FH's format handling is perfect. As one oddity, IIRC, when I first loaded my data into FH, my previous software had subfields for the personal name, splitting it into given name, surname, etc. FH loaded that data perfectly and processed it correctly from then on. It did, however, store the data as "John /Smith/" (which is the recommended format) and didn't update the given and surname subfields from then on. So if I updated "John Smith" to be "John Smyth", the Surname sub-field remained fossilised at "Smith". Logically, this didn't matter because it wasn't used - it's only nosey so-and-sos like me who saw this oddity when we opened the file. Still, a bit messy.

[tatewise]

John Cardinal's assessment is developed by him and offered for free. He has been adding further tests from time to time.
There are also more formal GEDCOM verifiers such as GEDCOM Validator.

Regarding such as PLAC.FORM, IDNO.TYPE, MARR.TYPE, etc, those fields are retained by FH so no data should be lost.
It is simply that FH does not use them to adjust the way data is displayed. Nevertheless, users can edit them.
However, it is often possible to customise FH to use those fields. See FHUG KB Recording a Civil Partnership under Fact Type Descriptor for an example.

The specific FACT.TYPE and EVEN.TYPE custom attribute and event definitions are fully supported by FH.

[davidf]

...
The PLAC and ADDR fields are associated via their parent Fact. ...

Except you are comparing apples and pears.

A PLAC (and its associated data) is associated with a fact through a link to a 0 @Pn@ _PLAC record which is unique in the .ged file - all GEDcom compliant programs having to ensure that every 2 PLAC ... line has a corresponding 0 @Pn@ _PLAC record? (Although why is the Gedcom line not 2 PLAC @Pn@ ?)

The ADDR association looks more like being just a subtag (merely a 2 ADDR ... line) of the Fact (albeit selected-able though FH's predictive entry) which is qualitatively different and to imply that that links a PLAC to an ADDR would fail rules of good database management? FH seems to hold internally during runtime a table of addresses built presumably from scanning all 2 ADDR ... lines and eliminating duplicates - but building that table has the massive assumption that those "duplicates" are duplicates. 2 ADDR St Mary's could be a house, a street, a district, a church, even a football stadium - and those lines refer to different "addresses" in as much as they are different locations (albeit the actual characters in the line are identical).

And that is the problem - until Gedcom has a 0 @An@ _ADDR structure (where the "n" is important to distinguish the street from the stadium etc. and where you can then store subsidiary information or link media etc. against the specific address) we have a problem and if FH wants to adhere to Gedcom it can only offer work-arounds?

The other issue that worries me is when people talk about combining Address and Place information into a single "record".

In general we understand a Place as an area - even if we might struggle to define the precise area of say a village and fall back on the parish boundaries. We would like mapping software to identify the area. Look at how Google Maps identifies:

• Provident Rhode Island - an area
• Huddersfield, West Yorkshire - a mapping pin on the Fruit and Veg aisle of Sainsburys

We tend to understand an Address as something closer to an entity that can be represented as a map pin although with large scale maps we recognise an address as an area. However a map pin close to the location of my letter box is an adequate representation of my address. But a map pin close to the location of the pigeon hole of the fruit and veg manager of Sainsburys in Market Street Huddersfield is is not an adequate representation of Huddersfield!

[tatewise]

David, GEDCOM 5.5.1 strictly compliant files do NOT support 0 @Pn@ _PLAC records. They are an FH extension to GEDCOM.
A few other products have something similar, but such Place Records are NOT a GEDCOM standard.
That is why they are linked by Place Name instead of Place Record Id so when exported to other products at least the PLAC field value is sensible.

By definition, all the subsidiary fields of a Fact such as DATE, AGE, PLAC, ADDR, NOTE are associated together.
That is how a GEDCOM text file is organised.

How FH chooses to represent that association in its internal database is another matter.
FH has chosen to list all Addresses in Tools > Work with Data as if they are dissociated from Places.
However, it could easily treat each Address as associated with a Place in the same Fact.
Then there would be distinct entries for say St Mary's for each Place it is associated with.

The mapping pin problem of trying to identify the centre of an area is well known and can only be resolved by highlighting the area instead of using a pin, but that is a completely separate issue.

[KFN]

Tatewise is correct, that PLAC has not been normalized to use XREF links, this is one of the drawbacks of being GEDCOM compliant!

If you follow GEDCOM-L proposals they extend GEDCOM by normalizing to _LOC records and XREF pointers for each location.

My hope was that GEDCOM 7.0 would have normalized PLAC but they fell short in many of my hopes.

[davidf]

David, GEDCOM 5.5.1 strictly compliant files do NOT support 0 @Pn@ _PLAC records. They are an FH extension to GEDCOM.
A few other products have something similar, but such Place Records are NOT a GEDCOM standard.
That is why they are linked by Place Name instead of Place Record Id so when exported to other products at least the PLAC field value is sensible.

Thanks - if that is the case we are looking for CP to enhance their PLAC extension and in respect of some of the issues around ADDR & PLAC we are not as constrained by "Gedcom compliance"!

Do we know the range of punctuation used across the world in location specifications? I'm wondering what the options are of having a single location (_LOC?) record, divided by some form of punctuation (not used in "real postal addresses" | ?) so that internally FH can present PLAC and ADDR and we can work on "multi-column places" etc. The entry process would have to be to select (by predictive entry?) an existing Place (or add new) then select (via pulldown?) an existing Address relevant to that address (or: add new, select Whole Place, or select a "Place Qualifier" such as "Registration District"). Each unique "location" can have a "map pin" based on the "Address" or some approximation for "Whole Place" (even if that is Fruit and Veg at Sainsburys).

[tatewise]

This is swinging back to the Places and Addresses (20441) topic that this compliance topic was branched from.
It is also related to the Wish List Ref 527 Add Media and other data to Address fields.
Some users were dismayed long ago that the FHUG Wish List for a Locations database was implemented as a Places database without involving Addresses.

I don't see any fundamental problem in having separate Address and Place fields that work together.
It is not unlike the Latitude and Longitude fields that are a working pair, or the many to one Citation to Source pairings.

[AdrianBruce]

... so that internally FH can present PLAC and ADDR and we can work on "multi-column places" etc. The entry process would have to be to select (by predictive entry?) an existing Place (or add new) then select (via pulldown?) an existing Address relevant to that address ...

Re "multi-column places". Before you get too eager to design stuff, bear in mind that some of us don't use multi-column places in the sense of trying to get everything into consistent columns.

I don't for two good reasons:
(1) It's far from clear to me what the consistent column arrangement for (say) "Gladstone, Queensland" and "Gladstone, Queensland, Australia" (say) would be (the difference is 1 January 1901, when the Commonwealth of Australia was formed). Is it putting the country and colony names in column 3? Or is it putting Queensland in column 2 in both?

(2) If I have a standard 4-node for the USA, with standard column positioning, then "USA" goes into column 4. But it would then need to be entered as " , , , USA". Except, I can never remember the ordering of commas and spaces. And, using the current predictive entry, I have to know that combination as " , , , USA" isn't the same as ",,, USA" (say). Ideally I should only need to type "USA" to pick up " , , , USA". That's simply a change to the predictive matching algorithm, but it does need to be thought about.

Also - a fairly restricted point - not having matching columns means that I'm at liberty to stick an extra column in my names for places in "Prussia, Germany" compared to those in other German states. Or to not stick pointless extra columns in front of "Hamburg", which was a City-State so is hardly likely to need many of those other German levels.

This all has implications for how Address and Place would get munged together - which may or may not be relevant.

[wildbill]

@davidf, thank you for your detailed post which is exactly what I expect.

I have considerable investment in Rootsmagic here and found two solutions which I hope will be temporary.

1. Direct import which preserves my Named Lists and Research items but dumps my important Address geocoding and other Site attachment information into Notes.

2. Gedcom import which dumps Lists and Research Items but allows me to further query my Address attachment data to further improve the quality of my database back in Rootsmagic 7.

Neither are satisfactory solutions to this shortcoming long term.

Whilst I do not care for the latest incarnation of Rootsmagic mapping of these sites was further enhanced and proved extremely useful in identifying communities and clusters in rural locations. It would appear FH just need to build a relational database behind the scenes to enable the enhanced data of sites and provide additional Gedcom export option for this non-standard Gedcom data but then I am not a programmer of any expertise.

For interest the extract below is how Rootsmagic apply the data in a non standard ADDR adaption

Code: Select all

2 PLAC Anytown, AnyCounty, AnyCountry
2 ADDR Harrington Street 3rd Presbyterian Church
3 MAP
4 LATI N49.5953389
4 LONG W0.9534306

[tatewise]

Interesting! That MAP, LATI, LONG subsidiary structure applied to ADDR is the same sub-structure as is applied to PLAC in GEDCOM 5.5.1 compliant data.

[davidf]

Re "multi-column places". Before you get too eager to design stuff, bear in mind that some of us don't use multi-column places in the sense of trying to get everything into consistent columns.
...
Also - a fairly restricted point - not having matching columns means that I'm at liberty to stick an extra column in my names for places in "Prussia, Germany" compared to those in other German states. Or to not stick pointless extra columns in front of "Hamburg", which was a City-State so is hardly likely to need many of those other German levels.

This all has implications for how Address and Place would get munged together - which may or may not be relevant.

I think I am more with you that not on this - but I am aware that how people work with FH can vary significantly and if we want CP to enhance some functionality, they need to feel not just that it is commercially worthwhile, but also that it will not bring a rush of complaints from users saying an enhancement has stopped them working the way they want to!

I don't make much use of multi-column places other than as a means to effectively sort places when doing a bit of "tidying" - so I tolerate the comma padding!

Another issue to me is having to avoid the "b****ing obvious". "Oxford, Oxfordshire, England" is pedantic overkill for me - and as for worrying about which county Bristol (or Belfast) are in!

I don't publish direct from FH (except for some diagrams), but use it as a structured source of data for when writing family history so I do not have to reconcile functionality in respect of things like sorting with "how it looks" when published.

I do wonder if an almost JSON like structure (Key: Value; syntax)might be appropriate to subdivide GEDCOM lines. Thus we might store in a "location" line:
"Church: 3rd Presbyterian Church; Street: Harrington Street; | Town: Anytown; County: AnyCounty; Country: AnyCountry;"

I already use this sort of format for GRO references and Census references in the "where used field" (I'm a lumper - still on V6). But it should be possible (in an era of templating!) to set up templates with relevant fields for locations (varying by country - allowing default entries in the "country" field?) to convert the above JSON structure into a "fill-in-the-blanks" form - where you fill in those blanks that are relevant but can still sort on the "county field" or the "street field" etc. where specified.

Thus you could configure a GB location form as:
Place (predictive entry):
Or New:
Town:
County:
Country:
--
Address: (Pulldown selection of addresses already associated with the above place)
Or New:
District/Village:
Parish:
Street:
Church:
Building No:
Building Name:
Postcode:

How much of this could be achieved as a plug-in to create the JSON syntax above (together with export plug-ins being able to remove the "Key:" characters and comma delimit the result)?

[davidf]

For interest the extract below is how Rootsmagic apply the data in a non standard ADDR adaption

Code: Select all

2 PLAC Anytown, AnyCounty, AnyCountry
2 ADDR Harrington Street 3rd Presbyterian Church
3 MAP
4 LATI N49.5953389
4 LONG W0.9534306

But what we are wanting is:

Code: Select all

2 PLAC Anytown, AnyCounty, AnyCountry
3 ADDR Harrington Street 3rd Presbyterian Church
4 MAP
5 LATI N49.5953389
5 LONG W0.9534306

?

[tatewise]

@davidf: GEDCOM 5.5.1 already has some Key: Value fields subsidiary to ADDR:
ADR1 Line 1
ADR2 Line 2
ADR3 Line 3
CITY City
STAE State
POST Postcode
CTRY Country

[davidf]

@davidf: GEDCOM 5.5.1 already has some Key: Value fields subsidiary to ADDR:
ADR1 Line 1
ADR2 Line 2
ADR3 Line 3
CITY City
STAE State
POST Postcode
CTRY Country

I am thinking very much of user definable keys (or a very wide variety of standard keys). For instance being able to put in "Parish" in that "hierarchy" (in inverted commas because of course it is not always hierarchical!) would allow sorts to assist with Parish Register Work. Likewise "Registration District" (which can span multiple counties). I also find that non-specific lines (like the first 3 above) can be a menace. If we want to look for neighbours a line "3 Church Lane" will sort with all the other "3"'s; "House No" and "Street Name" allows the separation for sorting by Street - but how do you ensure consistent positioning of data?

[tatewise]

User definable keys would be a menace to importing from & exporting to other products, obtaining Focus Window 'hints', and anything else that relied on location formats.
I have enough trouble with my 'Export Gedcom File' plugin coping with all the GEDCOM variant customisations in almost every product out there without adding even more

[davidf]

We throw out a fair amount of detail when exporting - which is why I was thinking about a single gedcom line with a series of Key: Value; pairs; exporting can strip out the Keys and comma delimit the remanent - which would then be very much like what we have in our existing PLAC and ADDR fields?

[wildbill]

I have looked at Heredis which also supports geocoding of subdivisions for mapping but play with the Gedcom standard in a different way.

Firstly this is the Heredis 2021 mapping and geocode support for the sub divisions;



And secondly this is their Gedcom output of the same data;

Code: Select all

1 PLAC
2 FORM Town, Area code, County, Region, Country, Subdivision
0 @1234@ INDI
1 NAME Michael/HARLOW/
2 GIVN Michael
2 SURN HARLOW
1 SEX M
1 EVEN
2 TYPE Residence
2 DATE 1 APR 1849
2 PLAC Newcastle Upon Tyne,,,,ENGLAND,Longbenton
3 MAP
4 LATI N54.973280
4 LONG W1.613960
3 _SUBMAP
4 LATI N55.008924
4 LONG W1.591601
2 _FNA NO

Regardless of how it is handled internally in the program I strongly believe this is a gap needing to be closed in FH as I believe Legacy Family Tree also provide support in some way for these important sites.

[KFN]

If you look at the GEDCOM-L design and proposal for “location” it is currently an add-on to GEDCOM, but can also be a replacement for the GEDCOM Place_Structure. It supports many of the things the current GEDCOM structure misses.

The _LOC tag is added at the same level as a PLAC tag:

2 PLAC Naustdal Farm, Naustdal Kommune, Sogn og Fjordane, Norway
2 _LOC @xref@

The XREF points at the lowest level entity (Naustdal Farm) which has a pointer to the next higher level, etc.

As a replacement it would just be:

2 PLAC @xref@

As part of the _LOC record it contains additional information about the location such as, but not limited to, Date of use, name, short name, long/lati, language of use, NOTE/OBJE/SOUR tags, type of location (street, cemetery, village, country, home, …).

The one thing that it does not do, and I’m very disappointed, is use polygonal boundaries rather than long/lati so one can map the size and shape of (for example) country borders. I’m sure some will think of a few other things that can be added, and it has a few things I’m not sure how I would use or maybe solve something I don’t see as a problem.

The point for me is that it eliminates the need to carry the Address_Structure as a location and better apply it for use as a postal addressing structure.

Because each record can point to a higher level, a location record can have multiple names, for example a city that has a German name and a Polish name depending on either date or resident ethnicity. I find this valuable in many places where land was controlled by or enumerated by different countries over time or where various ethnic groups used different names for the same place (city, battle, …). In the same way the location can have different long/lati for the same name based on date, for example if a church moved, if a polygon was used we could see the size of a city, country, farm, other entity change over time.

With the “next level pointers” you could skip some subdivisions, such as American townships or even counties/Boroughs if you don’t know them or don’t care to document them and change the relationship at a later date.

Davidf said

I do wonder if an almost JSON like structure (Key: Value; syntax)might be appropriate to subdivide GEDCOM lines. Thus we might store in a "location" line:
"Church: 3rd Presbyterian Church; Street: Harrington Street; | Town: Anytown; County: AnyCounty; Country: AnyCountry;".

GEDCOM already has this in the form of the PLAC.FORM tag, seldom used, and rarely supported. Using this would eliminate the need for multiple place holder comma (,,,,USA or "Gladstone, ,Queensland") or writing

2 PLAC <name> Farm, <name>Kommune, <name> Fylke, Norway

Vs.
2 PLAC <name>, <name>, <name>, Norway
3 FORM farm, Kommune, Fylke, country

Or in your case:
2 PLAC 3rd Presbyterian Church, Harrington Street, AnyTown, AnyCounty, AnyCountry
3 FORM Church, Street, Town, County, Country