* Export GEDcom plugin -- some questions

[ColeValleyGirl]

Mike,

Some questions if I may.

1. Is there an option (or some other way) to tidy Place records -- i.e. remove leading and trailing spaces so (e.g.) " ,Birmingham, England" becomes "Birmingham, England".

2. Is there anything proprietory about the format you use for the GST export to document fact customisation/custom facts? How fixed is it? Can I use a GST export for my own purposes confidently?

Thanks

Helen

[tatewise]

1. By default in most exports the Plugin removes all redundant commas/spaces from Place and Address fields.
   That includes any series of leading, central, or trailing multiple commas/spaces, except where needed to separate parts.
   However, an exception is made for FH5, FTM, GST & TNG where no tidying is performed.
   Obviously, exports to FH5 must keep everything. FTM is a special case.
   Exports to GST and TNG retain Place records that link by unique name, so changing the Place names is not allowed, as tidied names may no longer be unique.
   
   I'm guessing you want to use the GST export format and tidy Place fields too, so the only alternative is to run a global Search & Replace on the exported Gedcom file, but it is not straightforward, especially for the trailing commas/spaces, and you need to check the resulting names are unique.
2. The GST custom tag format for Fact Definitions was agreed between me and John Cardinal.
   I don't think there is anything proprietary about it, but it would be courteous to inform him of your proposed usage.
   The format has not changed following the initial design, and seems unlikely to change, but I cannot guarantee that indefinitely.

As an afterthought, it might be possible to concoct a special variant of the Plugin for the GST format with tidied Place names that you keep for yourself and manage the risk of non-unique names.

[ColeValleyGirl]

Thanks, Mike -- I will have to review how I handle the places. I may even have to develop my own export plugin and format for the customisation data (that also addresses the place issue) -- I'm intending to modify GEDmill (which is open source licenced) to produce a website more to my tastes than the layout FH produces. Initially the product would be for my personal use (plus my partner's) but if it works well there's no reason I couldn't give it away -- which would be in conflict with John Cardinal's commercial needs. (It wouldn't be anything like as full-featured as Gedsite, but even so it wouldn't be fair to build on the work he has done with you and then compete in any way with his product.)

[lastejas]

Mike,

Not sure what you mean by this statement re TNG:- "Exports to GST and TNG retain Place records that link by unique name, so changing the Place names is not allowed, as tidied names may no longer be unique."

At the moment I use you excellent Plugin but then manually remove the extra commas and spaces fron the exported file before importing into TNG (very time consuming as I have this on every export). Upto now I have had no problems with TNG so it would be nice if there could be an option to tidy up the place details in the plugin.

Thanks

[tatewise]

Helen,
Maybe what is needed is an export option for GEDmill that uses the Fact Definition format, with at least the Standard GEDCOM 5.5 settings to cater for custom FH tags, which remove Place records and tidy Place names by default.
I could look at that for the next version.

Rick,
The problem resolves around the way Place fields link to Place records using the Place Name.

Other links use the Record id that is guaranteed to be unique, so the Name/Title of the record is irrelevant and can be the same in different records.

But it is the Place field Name that identifies the linked Place record, and that Name MUST be unique.

So imagine there were two similar Place field Names :-
, New Town, England, and New Town, , England that link to two similarly named Place records.

If they were tidied then all those Place field & record Names would become identical :-
New Town, England
So there would be two Place records with the same Name and the linkage between Place fields and Place records is upset.

So in general the Plugin cannot perform the tidying.
It would have to monitor every tidied Place Name and complain if two Place records got the same Name.
Then should it refuse to produce the exported Gedcom or what?

[lastejas]

Mike,
Thanks for that explanation and I can see where your coming from, but from your example wouldnt that be the case no matter what software you were exporting to?

So to clean up the place records, I would have to use the plugin to export to say RM (as thats not in your exception list) and then import that gedcom into TNG.

[tatewise]

No, the problem only applies to products that support Place records, i.e. GST and TNG.
The others can tidy the Place field names, because the Place records are removed, so no name conflicts exist.

If you export to another product such as RM then the exported Gedcom will be tailored for RM and not TNG.
So none of the special features supported by the TNG export option will be available.
To understand the details of the export options for each product you need to investigate the Help & Advice for the Output Formats for each product.

It is interesting that nobody else has mentioned the Place name tidying problem with TNG.
Ron Krzmarzick (ronk) is an extensive user, and will ask him to consider this issue.

I also could investigate the technique I suggested, to monitor tidied Place names, and only baulk if there is a conflict.

[lastejas]

Thanks for that Mike and will read through the information again.

Cheers for all you help, much appreciated

[ColeValleyGirl]

Mike, hold fire on 'GEDmill support'; the work I'm thinkg about doing depends on a Proof of Concept for diagrams that somebody else is doing... so I'm just at the stage of exploring what is feasible and how I might do it and how much work is involved.

[tatewise]

I always like a challenge, so I have taken a quick look at tidying Place names when Place records are involved.
It does not seem too difficult, and if a Place name conflict arises it can simply not tidy the offending Place name, and report such cases.
But in most databases there will probably be no such cases and every Place name will tidy OK.

I was thinking other users might like to use GEDmill and that the Gedcom dialect would be the same as your version Helen.

[ColeValleyGirl]

If we can contrive a simple export format for GEDmill (to match it's simple ISR-based output) that might help a number of users who would like little navigation trees in their website, and also those who prefer to see marriage information in chronological order.

[tatewise]

Try that attached Export Gedcom File Plugin ZIP file that must be extracted. [Now Deleted]
It offers my first stab at a GEDmill export option and adds some toggle options to the bottom of Extra Options tab.

[ronk]

Try that attached Export Gedcom File Plugin ZIP file that must be extracted.
It offers my first stab at a GEDmill export option and adds some toggle options to the bottom of Extra Options tab.

OK Mike, I see V3.3 chg line (vs. 3.2.1 in version line) has quite a few items including ASID orphans, and Tidying Places. Will give a try later today or tonight and report back.

Thanks!

[tatewise]

Ron, internally I record interim version changes as if for the next published version, which will be V3.3.
Not only is there that @V3.3: summary, but wherever the script is changed it is commented with -- V3.3
That saves me going through and amending them all when it gets published.

Helen, I can't help making some observations on GEDmill.
Generally, it is exceptionally quick and thorough, but there are a few oddities, even with some standard Gedcom items.

1. Usually in Facts, any Address is added in parentheses after the Place, but for a Marriage it produces and extra preceding 'Fact' with just the Address and other fields a replica of the Marriage entry.
2. The Age is omitted from all Facts.
3. It accepts Custom EVENts with Attribute values despite not strictly being Gedcom 5.5 compatible.
4. All LDS Ordinance Facts are omitted, as are many minor data tags such as ALIA, ASSO, REFN, etc.
5. The Address field is omitted from Repository entry at the top of Source displays.
6. The Author is omitted from Source displays, and only one of Title or Short Title is shown when both exist.

The Export Gedcom File Plugin adjusts for all but the first and last of those.

[ronk]

Try that attached Export Gedcom File Plugin ZIP file that must be extracted.
It offers my first stab at a GEDmill export option and adds some toggle options to the bottom of Extra Options tab.

Mike, did a comparison of Export Gedcom 3.3 and 3.2 (LFT option) with a full text comparison utility to identify all differences, with no settings changed from what I have been using.

One step forward, and one step backward for me on a couple items. Yes, the ASID orphans are removed, but unfortunately, the 'caption note' label or tag has returned which at one time I was removing manually, as I was recently the ASID records. I compared the settings and the ony changes I saw in 3.3 were the ticked box for places/addresses, and insert synthetic prefix at the bottom, vs the 'tidy address field on the center right in 3.2. Do you remember which setting removed the 'caption note' text? I have to go back to our posts exchanged last year.

As to address tidying with 3.3, I only saw a few extraneous spaces removed, and no place records are exported.

I will test the TNG export tomorrow.

[ColeValleyGirl]

Mike,

Thans for this -- it's good to be able to exclude [[private]] information, to tidy the places and handle second places on the emigration event, and add ages; plus being able to add the fact defiitions to the export is a plus. I still need to tinker with the default options to work out how to produce the best results. (But... "Lady" partner! Really? what's wrong with female partner... or Even Partner 1 and Partner 2.)

On your comments on GedMill, I think it was only ever intended to be a simple website generator with graphical tree navigation, so it is missing a lot of bells and whistles... The extra address fact for the marriage is (as you note) odd and I can't remember seeing it in earlier versions so I'm thinking it might be a bug. Some of the other points you mention could be addressed if I decided to do a major overhaul of it.. but I'm still assessing the work involved versus the payback, and assessing GedSite.

[tatewise]

Helen, I chose those Age related labelled Note formats to be the same length so they would tabulate neatly:-
Male Partner Age:
Lady Partner Age:
Individual's Age:
To be honest I am surprised how many products do not support the standard fact AGE tag:-
Ancestry, FindMyPast, GedStar Pro, Roots Magic, TribalPages, and now GEDmill.

Ron, I have investigated the _ASID and Caption Note with FILE~ABS/REL links with the following analysis.

When _ASID were included, then all the captions were included as 1 NOTE tags in the Media records.
But each caption was in a new instance of the 1 NOTE tag, which is the FH Media format for Link/Note entries.
My thinking behind FILE~ABS/REL links was that they would need FH dialect Gedcom (but I was wrong).
I suspect LFT only supports the 1st instance of each caption, so only the 1st caption was visible.

The revised version omits all the FH custom Media tags such as _ASID, _AREA, _EXCL, _CAPT.
For multiple 1 NOTE Link/Note captions it combines them into a single instance multi-line 1 NOTE labelled Caption Note.
In cases where the captions are all different, then that works well, and none of the captions vanish from the exported Gedcom.
But yours is a special case, where all the captions are identical, so they get repeated, which I agree is undesirable.

So, my current plan is to detect such identical repeated captions, and omit all but the first one.
That maintains my ideal of retaining all the Link/Note captions, but only if they are different.
To keep it manageable, the rule will be that they are omitted if the captions are identical and less than 250 characters long, such that they fit into one Gedcom line. Will that satisfy your needs?

[ColeValleyGirl]

Mike, doesn't neat tabulation depend on the font the text is displayed in?

[tatewise]

Yes, it does, but since only three characters differ, and the labels followed by a tab character, they are more likely to tabulate neatly than if I used Female instead of Lady.

[ColeValleyGirl]

No problem -- I can always fix the Gedcom before I import it into GEdmill, or fix the HTML afterwards.

[ronk]

So, my current plan is to detect such identical repeated captions, and omit all but the first one.
That maintains my ideal of retaining all the Link/Note captions, but only if they are different.
To keep it manageable, the rule will be that they are omitted if the captions are identical and less than 250 characters long, such that they fit into one Gedcom line. Will that satisfy your needs?

Mike, does that mean only one Link/Note caption remains for the entire Gedcom? Or for each instance of Link/Notes? I removed all of them when they were present before and wish to have none in the Gedcom file. Also, tried the TNG option, and see the same issue (for me) in that rendition as well. If it's too specific to me or too difficult I can do it manually.

[ronk]

To keep it manageable, the rule will be that they are omitted if the captions are identical and less than 250 characters long, such that they fit into one Gedcom line. Will that satisfy your needs?

In the Gedcom for LFT and TNG, here's an example (also contains ASID orphans, which I remove manually:
BEFORE v3.2:
1 NOTE Daniel Getty,age 6, and Robert Getty family, likely match.
2 CONT
2 CONT Philadelphia, 1800-1850 Passenger and Immigration Lists
2 CONT about Robert Getty
2 CONT Name:Robert Getty
2 CONT Arrival Date:31 Jul 1833
2 CONT Age:36
2 CONT Gender: Male
2 CONT Port of Arrival:Philadelphia
2 CONT Place of Origin:Antrim

AFTER:v3.3:
1 NOTE Caption Note: Daniel Getty,age 6, and Robert Getty family, likely match.
2 CONT Caption Note:
2 CONT Caption Note: Philadelphia, 1800-1850 Passenger and Immigration Lists
2 CONT Caption Note: about Robert Getty
2 CONT Caption Note: Name:Robert Getty
2 CONT Caption Note: Arrival Date:31 Jul 1833
2 CONT Caption Note: Age:36
2 CONT Caption Note: Gender: Male
2 CONT Caption Note: Port of Arrival:Philadelphia
2 CONT Caption Note: Place of Origin:Antrim

So you were suggesting to remove all but the first 'caption note' text, right? Well, that text, even just one, is redundant and unneeded in LFT and TNG, the way it appears (for me) in those products. So either way, with 3.2 I need to remove ASID manually, or with 3.3 remove all 'caption note' labels.

[tatewise]

We seem to have got our wires crossed.

I was working from your posting Re: Export Gedcom ASID tag Errors in LFT where only duplicated caption notes appeared to be the problem.
So to fix that I was proposing to reduce those duplicates to just one Caption Note.
That is what I mean by "they are omitted if the captions are identical".

Whereas now it seems you want no Caption Notes at all, at all.

There are also possible Picture Note and Keywords labelled Notes on Media.
You may not use them, but would you want them removed too?

[ronk]

Whereas now it seems you want no Caption Notes at all, at all.

There are also possible Picture Note and Keywords labelled Notes on Media. You may not use them, but would you want them removed too?

Mike, In that post you referred to, Export to Gedcom v3.2 was discussed, and, with the settings chosen, the output from it had zero caption note labels on any link/notes, not just duplicated ones. That's why I was surprised when you indicated v 3.3 would keep those that were not duplicated since with v3.2 there were none at all, which was what I wanted. Those labels look redundant in LFT and TNG since they have there own section headings.

As to the other items, I'm fine with whatever 3.2 was doing, and but as you've done, removing the ASID orphans for LFT and TNG.

Sorry if there was confusion on this point.

[tatewise]

OK, try the attached Plugin ZIP file. [Now Deleted]
It is a little inelegant in parts, but I think it will offer what both Ron and Helen require.
The difficulty is that the Plugin primarily works on Gedcom tags, but the NOTE tag is used everywhere and specific functions for different contexts are awkward to govern with the Plugin current design.

The new options are on the Extra Options tab, about four lines up from the bottom.
Namely Record Note 1 NOTE: and Subsidiary Note %d NOTE:

Ron, try them with default setting of Media Caption Note and LMO Caption Note respectively to include all Media Notes.
Then try them with Omit Media Notes and LMO Caption Omit to omit all Media Notes.
The FH custom Media tags _ASID, _AREA, _EXCL, _CAPT are always omitted except for FH5 and GST exports.
There are separate options for Picture Note %d _NOTE and Media Keywords %d _KEYS and Media Date %d _DATE.
But if they are set to Move to local Note with the Omit options above, then they will also be omitted.
Try them with both LFT and TNG.
I am a little uncertain about the impact on the Keywords _TYPE feature for TNG but it should be OK.

Helen, for GST to get Marriage Status and Family Record Notes into a custom fact use the following Extra Options:
Marriage Status 1 _STAT: Move to Record Note near top right.
Record Note 1 NOTE: Move to Fam Note Fact lower left.
That should create a custom Family event named Fam Note with the Note field containing all Family Record Notes and the Marriage Status as a labelled Note.