* Ancestry, FTM, FH and workflow

Importing from another genealogy program? This is the place to ask. Questions about Exporting should go in the Exporting sub-forum of the General Usage forum.
avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 05 Apr 2015 20:06

Yes its hard sometimes :) Simpler on the phone if necessary. Plenty of room for misunderstandings, frustrations and lack of answers...just the way it is. Despite almost obsessive input of time not everything is certain or tested - and nobody has all the answers. It's just you and me, unforeseen obstacles should be expected and perhaps its a little more involved than anticipated! Nevertheless, we've delivered, we're still moving forward and I see no show-stoppers.

Use of Description field for Address and Value etc.:
Yes formatted note tags for all events/attributes/facts no matter what and even if there is no value as this is the master data for the Import (use of the Description is a pleasant extra). You need to know which is an event and which is an attribute so I am simply saying mark them in the note (not in the Fact Description) and mark them discretely - not something big like "_ATTR". Or if you don't need to then forget the marking idea - again just responding to what I though you were meant. Any further uncertainties let's speak on the phone or just do it and see what happens. The documenting is unduly tiring and distracting us...

Lists and Preserving PLACe record data:
I've not found submitter records or similar in FTM. Source is the likely solution. Was just wondering if any note (local or otherwise) can support a child OBJE - like 0 NOTE or like citations but I don't think we need pursue further my attempt to hide this data.

From memory the linking is to required to make the Source item usable so I linked my few Sources to a fake INDI - no idea how it affects round trip. What I am highlighting is that linking to real INDIs causes the NOTE to be excessively cluttered. So, either 1) we don't need to link the SOURce at all to survive a round trip,or 2) if we do then we need to link it to something that does not cause the note problem. I will put this on the list of tests and outstanding things to think about.

After 2 hours of web research I came up with Ω because the normal characters don't sort to the end including {|}~. I've tried many of those characters - I have no answer other than ZZZ (which is nasty) or further research.

Witnesses...OK that's a welcome relief.

Record numbering is a research matter on its own. I believe FTM has something about associated people MAYBE, I know nothing of the other records yet...but I'm not going to give in to numbering and total replacement just yet (even tho it may well be necessary) until we've established a few more facts.

I'll try again to start Media folders research.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 05 Apr 2015 21:18

I think the FTM Media sub-folders matter is only of passing interest, because the Plugin uses a single folder and the labelled Note details define the original sub-folder in FH so it can be reconstructed on return. I have no intention whatsoever to export Media in sub-folders.

I think the other matters are far more pressing.

Use of Description field for Address and Value etc.:
I have no intention of creating label entries in Fact Notes when there is no associated data value.
That would be like creating every possible tag in every possible Fact even when it has no data value.
The one exception will be EVENts derived from _ATTRibutes where it is needed as a differentiator.

Lists and Preserving PLACe record data:
NOTE tags in any position, including 0 NOTE records, do NOT support OBJE links, so we will stick with SOURce Records.
Due to the FTM disruption, I will allow Named Lists to use SOURce Records instead of NOTE Records at least for FTM.
I will allow synthesised SOURce Records for Named Lists, Places, etc, to have Title prefixed with a specific character to enhance sorting.

The jury is out on Record Renumbering, but I favour adding [Rec Id] to a labelled Note or the end of Record Title/Name.
Probably a Note is best for INDI, FAM, SOUR, OBJE & NOTE records and suffix on Title/Name for others that don't allow Notes.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 05 Apr 2015 22:01

Use of Description field for Address and Value etc
I'm responding to a point I thought you made and just can't imagine what you think I'm trying to get you into. Let me know if you want my phone number.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 06 Apr 2015 00:46

I've found the cause of death appearing not only as it should (event and note) but also as the cause in a note against their spouses birth event. Notably the position of the spouse was one before the person with the valid cause.

I wonder if this just affects cause of death.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 06 Apr 2015 21:31

I should have suggested this earlier, and maybe you already do it:
Copy the Family Historian Sample Project to say a Test FTM project.
Then add as many test cases for as many tags in as many records as possible.
The benefits are the Plugin exports very quickly and it's easily edited without disrupting your master Project.

If you use that technique and add CAUSe to many Facts you will soon see what happens.

I have progressed well with new updates to the Plugin.
Can you review this list to see if I've missed anything.
  1. Extra Options for NICK & _USED move up to 2nd row below _LIST and _ROOT options.
  2. File Root and Named Lists now optionally can be Source records.
    These are linked directly to listed Individual, Family & Note records.
    They are linked via a local Note to listed Source, Repository & Media records.
  3. All synthetic Source records have Omega Ω prefixed to Title in any Unicode mode.
    Have tricked ANSI Plugin to produce UTF-8/UTF-16 Ω code, but yields O in ANSI/ISO mode.
  4. Individual Attributes with a Value have Attribute Value: labelled local Note.
  5. Custom Events from _ATTR always have Attribute Value: labelled local Note.
  6. IDNO & NCHI become a Custom Event with Value and labelled Note as above.
  7. CENSus (family) becomes a Custom Event.
  8. Custom Events from Record Flags now use 1 EVEN Y.
    Alternatively, the Record Flags can become Record local Notes.
  9. Custom Id (REFN) in Source & Family records become a Record local Note.
  10. Record Website (1 _WEB) now appends to E-mail in Repository records.
  11. Media Format (1 FORM) will now Move to Record Note.
The two I have on back-burner is moving Repository tags with no FTM Field to a Source record, and including Record Id in every exported record. Can you think of any others?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 07 Apr 2015 00:52

Its late so I'll get to running thru the list and KB step 2 with a fresh head on Tuesday - sound good tho. I guarantee we've missed several things...we should foresee the unforeseeable.

For testing I use a copy or two of my live data and add additional data to it for testing, FTM is OK with several trees. Sometimes I simply edit the GEDCOM. To trace this particular error one might use known data (happy to send my GEDCOM) that can recreate it and step by step that particular bit of code. However, I've no idea what animation/debug/stepping facilities you have with the LUA setup.

Thinking about the one example I accidentally found, I remember your approach is to read all the lines of a record, manipulate and write it out that record...it sounds like the next record is putting CAUS into the Description a little too early (whereas if it were the previous record I would have suspected a field was not being initialised). As nothing else jumped out at me I guessed it could be CAUS...but of course it could be more...

Certainly I can chuck lots of data at it and have another guess but wonder if there is a more refined way first that is quicker as we should be able to recreate it. Time is of the essence...the lack of it stinks.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 07 Apr 2015 08:30

Please give some more precise details about the CAUS problem.
At present I cannot understand how data can migrate to a preceding record.

This morning you say it's "putting CAUS into the Description", which is on same line as the Event.

Yesterday you said it's putting "the cause in a note against their spouses birth event".

Those statements are either contradictory, or both true but economical with the explanation.
A birth event should only ever have an Address in its Description.

Exactly what format of cause text are you seeing in both the Birth and Death events?
If in the Description it should just be the CAUS value as in the FH Project.
If in the Note it should be labelled as Cause of Fact: and then the value.

Are you sure the Birth Event does not have a CAUS value in the FH Project?
You will have to look on the All tab to see it.

LUA has very good step by step debugging features.
Al you need to post is the fragments of Gedcom for the problem events, both in the FH Project and in the exported Gedcom.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 07 Apr 2015 08:33

Yes I see the contradiction...can't remember, will check.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 07 Apr 2015 20:56

This is strange but here is the data from the Plugin then FH. The first observation is that there is a CAUS tag against BIRT in the FH Gedcom so it's not the plugin. This tag also displays in FH's expanded record list yet I couldn't create a CAUS there if I tried. Clearly there has been an earlier data hiccup not related to the Plugin...I wonder what else!

Then I also note the 3 SOUR against BIRT that does not exist in the FH GEDCOM. After a while I thought, of course this is because the Plugin created Sources and links to them for places.

Mike...I don't see anything for you here except a strange circumstance to be aware of.

EDIT: I suddenly wondered if I had create this line somehow by editing the GEDCOM during testing. I confirm this is not the case as the data anomaly also appears in my earliest surviving backup dated 06-04-2014 - long before any GEDCOM fiddling.

Plugin GEDCOM

Code: Select all

0 @I176@ INDI
1 NAME James /Love/
1 SEX M
1 BIRT
2 DATE 27 MAR 1791
2 PLAC Long Ditton, London, England
3 SOUR @S445@
2 NOTE Place Details:	Long Ditton, London, England
3 CONT Cause of Fact:	Decay of nature
.......more
0 @I177@ INDI
1 NAME Sarah /Leader/
1 SEX F
1 BIRT
2 DATE 1793
2 PLAC Old Brentford, London, England
3 SOUR @S446@
2 NOTE Place Details:	Old Brentford, London, England
1 BAPM St Mary's
2 DATE 15 FEB 1795
2 PLAC Ealing, London, England
3 SOUR @S382@
2 NOTE Place Details:	Ealing, London, England
3 CONT Address Data:	St Mary's
1 OCCU Greengrocer
2 DATE ABT 1841
1 DEAT
2 DATE 4 AUG 1857
2 PLAC Isleworth, London, England
3 SOUR @S408@
2 CAUS Decay of nature
2 NOTE Husband James present at death.
3 CONT Place Details:	Isleworth, London, England
3 CONT Cause of Fact:	Decay of nature
FH GEDCOM

Code: Select all

0 @I176@ INDI
1 NAME James /Love/
1 SEX M
1 BIRT
2 DATE 27 MAR 1791
2 PLAC Long Ditton, London, England
2 CAUS Decay of nature

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 07 Apr 2015 21:49

You mentioned the long list of further enhancements in progress...good news. I look forward to testing and seeing them in action.

I try to ensure that all known development matters are at least listed in KB Step 2 rather than scattered around the forum where they may have been discussed, sometimes in greater detail. As I reread the KB the only things that jump at me are:
  1. _ROOT seems unnecessary, I feel it should not be output at all. The first INDI is the ROOT and this might change before the return trip rendering any data as incorrect.
  2. Thanks goodness for the Omega trick.
I'm certain there are a few other fields, other bits of processing like media folder, unforeseen FTM "wonders" and simple errors that remain. However, I think we can fairly say that the majority of FH data (and all info necessary for matching) can move to FTM sensibly and in a manner likely to support a return trip (given an appropriate import plugin and subject to minor enhancements to cater for unforeseen issues arising).

How has the plugin survived this bastardisation? Is it still maintainable or has it become a little unwieldy?

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 07 Apr 2015 22:13

I had a vague recollection of a problem with CAUSe some time ago, and I have found the proof:
See fhugdownloads:contents:non_death_facts_with_causes|> Query:Fact ~ Non Death Facts With Causes.

File _ROOT has an option to remove entirely if you wish (it still exports the File _ROOT INDIvidual first).
Other users may want to restore the original FH File _ROOT regardless of the order of the INDIvidual records returned from FTM.

If the export is in ANSI then Omega is replaced with ZZ.

The Plugin now puts the Record Id into every record, and moves the Repository tags without fields to a Source record.

Can you investigate what FTM does with SUBMitter and SUBmissioN records.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 07 Apr 2015 22:37

Ah ha...most interesting - known issue...I'll run the query.

EDIT: the download link fails. The alternative is use the All Facts query, add CAUSe field, exclude is %FACT.CAUS% is nul - also exclude DIV and DEAT events if you wish...I simply sorted the output.
MIKE'S EDIT: I have informed Jane about the faulty download, which is, it turns out, one of a number.

OK, I'll try to learn what they are all about and get back with some info.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 08 Apr 2015 09:25

SUBM/SUBN - summary: it appears FTM does nothing although it throws import errors.

2 SUBM records created and 2 SUBN records created both with links to the first SUBM. The 2 SUBMs exported but only 1 SUBN. Notably it appears FH allows creation of multiple SUMN but only saves one to the GEDCOM. I note how tags have been renamed, deleted and appended in line with other processing.

FH Data is:

Code: Select all

0 @U1@ SUBM
1 NAME Subm Name
1 ADDR Subm Addr
1 PHON Subm Phone
1 _EMAIL Subm EMail
1 _WEB Subm Web
1 LANG Subm Lang
1 RFN Subm Regn
1 CHAN
2 DATE 8 APR 2015
3 TIME 10:11:24
0 @U2@ SUBM
1 NAME Subm2 Name
1 ADDR Subm2 Addr
1 PHON Subm2 Phone
1 _EMAIL Subm2 Email
1 _WEB Subm2 Web
1 LANG Subm2 Lang
1 RFN Subm2 Regn
1 CHAN
2 DATE 8 APR 2015
3 TIME 10:04:57
0 @B1@ SUBN
1 SUBM @U1@
1 FAMF Subn File
1 TEMP Temple Code
1 ANCE 10
1 DESC 9
1 ORDI no
Plugin data is:

Code: Select all

0 @U1@ SUBM
1 NAME Subm Name
1 ADDR Subm Addr
1 PHON Subm Phone, Subm Web
1 EMAIL Subm EMail
1 LANG Subm Lang
1 RFN Subm Regn
0 @U2@ SUBM
1 NAME Subm2 Name
1 ADDR Subm2 Addr
1 PHON Subm2 Phone, Subm2 Web
1 EMAIL Subm2 Email
1 LANG Subm2 Lang
1 RFN Subm2 Regn
0 @B1@ SUBN
1 SUBM @U1@
1 FAMF Subn File
1 TEMP Temple Code
1 ANCE 10
1 DESC 9
1 ORDI no
FTM reported 4 import errors (the 2 below twice):

Code: Select all

Line 49813: error 4  : Unsupported or invalid tag: LANG. Line ignored.
Line 49814: error 4  : Unsupported or invalid tag: RFN. Line ignored.
FTM Displayed:

Code: Select all

Cannot find anything
FTM exported (which it always does even when no SUB records are imported):

Code: Select all

0 HEAD
...more
1 SUBM @SUBM@
...more
0 @SUBM@ SUBM
0 @I56@ INDI

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 08 Apr 2015 16:41

With Jane's assistance the fhugdownloads:contents:non_death_facts_with_causes|> Query:Fact ~ Non Death Facts With Causes download now works. A few others needed fixing too.

The Export Plugin now creates Source Records to hold the SUBM/SUBN Records for round trip.

I am not sure whether allowing multiple SUBN, but only one SUBN saved in Gedcom, is a fault or what.
EDIT: Gedcom says only one SUBN is allowed, so FH should not allow extra ones to be created.

Nearly every type of record allows Custom Id 1 REFN and a subsidiary tag of 2 TYPE.
We know some don't like REFN, so for consistency should they all become Record Notes?

INDI records allow Permanent Ref No 1 RFN.
INDI & FAM allow Submitter links 1 SUBM @U1@.
Have you tried importing any of those to FTM?

How is Child Relationship handled on Family as Child link in INDI records on import to FTM?
i.e.
1 FAMC @F2@
2 PEDI Adopted

I presume any UDF in an FH Project should be resolved before exporting to FTM.

You asked how the Plugin is standing up to the FTM additions.
So far it has survived reasonably well without major changes to strategy.
The whole thing, like most of my other Plugins, is driven by tables and rules.
For example adding extra rules for FTM only affects the FTM mode, because the rules are never even created for other modes, so they don't slow down exporting in those modes.
Also the Extra Options tab automatically adjusts as new rules need to appear, because I knew when building the Plugin that extra rules were bound to be needed.
Most of the rules simply invoke a small self-contained function, so FTM needs a few more than most.
There are a number of shared service functions, such as moving stuff into Notes, and some of those have needed amendment.

One thing that may need investigating is your slow initial run time.
To keep it simpler, the Plugin sometimes creates dummy records & tags using FH Gedcom, and expects other rules to convert to FTM compatible Gedcom, which means double processing.
For example, when copying Source Media to Citation Media it literally copies the OBJE related tags from Source to Citation, including all the _ASID, _EXCL, _CAPT, etc. Then these get translated using all the Media conversion rules.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 10 Apr 2015 20:42

Sorry for a late reply...I'm sure the posting was shorter when I first looked...

Thanks for feedback on SUBN. Some overkill there I think!

Yes REFN to record note for consistency sounds good.

There are three matters I need to learn about and investigate...what is the UDF matter?

Your info regarding the plugin was interesting thanks. The terrible slow matter is not something I've thought about recently for some reason...I'll take note if it hits me badly.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 10 Apr 2015 21:06

The notes below are intended to respond to the questions regarding INDI, RFN also INDI & FAM, SUBM also FAMC, PEDI. Also I added 2 FAMCs - I happen to know links to several parents can occur in FTM. I think we already know SUBM and SUBN data does not appear to import to FTM.

Plugin exported (edited):

Code: Select all

0 @I3525@ INDI
1 NAME Ian Gerald /Smith/
1 SEX M
...more
1 FAMC @F31@
2 PEDI Foster
1 FAMC @F3@

0 @I4548@ INDI
1 NAME 0 Fake person for source citations /?/
...more
1 SUBM @U1@
1 RFN 123456789
1 REFN CustomID

0 @F31@ FAM
1 HUSB @I57@
1 WIFE @I14@
1 CHIL @I17@
1 CHIL @I3525@
1 SUBM @U1@

0 @U1@ SUBM
1 NAME Subm Name
1 ADDR Subm Addr
1 PHON Subm Phone, Subm Web
1 EMAIL Subm EMail
1 LANG Subm Lang
1 RFN Subm Regn
On import we got the expected errors for SUBM, LANG and SUBM, RFN (but not INDI, RFN).

Inside FTM I could see: TBA

This is the FTM export file excerpt. Note PEDI has gone (I believe FTM uses _FREL and _MREL), two FAMC tags remain, SEX U has been create, RFN has gone, REFN remains, SUBM record and links have gone:

Code: Select all

0 @I1@ INDI
1 NAME Ian Gerald /Smith/
1 SEX M
...more
1 FAMC @F3@
1 FAMC @F30@

0 @I3163@ INDI
1 NAME 0 Fake person for source citations /?/
1 SEX U
...more
1 REFN CustomID

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 11 Apr 2015 12:25

I have posted an updated Plugin V1.7.9 in the KB.

To answer your question, UDF are glossary:udf|> Uncategorised Data Fields and can be fixed as explained in how_to:handling_unrecognised_data_fields|> Handling Unrecognised Data Fields.

The latest Plugin takes a new approach with the HEAD Record. It copies all the tag lines to a Source Record, which after the round trip, allows the HEAD Record to be reconstructed. This means that any unwanted tag lines in the HEAD can be Removed Entirely, such as File Root and Named Lists.

All the recently investigated minor tags such as REFN, RFN, AFN, RIN, PEDI, _PEDI, SUBM links, and few others, now become Record local Notes.

The Record Id is put in every Record local Note except for Repositories where it's appended to Name.

SUBMitter and SUBmissioN records become Source Records.

The updates 1. to 11. listed last Monday are all included.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 11 Apr 2015 12:38

OK, sounds comprehensive = a lot of work done by you and a lot of testing and some documentation required now! I'll try to address it over the next day or three. Surely, there's not much that doesn't export now?

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 11 Apr 2015 14:26

Please refer to KB Step 2.
[ MIKE's comments in brackets. ]

UDF
The currently known issues resulting in UDF when importing from FTM to FH are few and as follows: REPO, EMAIL and then CONC child tags on REPO, NAME and REPO, PHON and REPO, EMAIL and INDI, OCCU and INDI, EDUC.
[ CONC & maybe CONT can probably occur on any Fact with a long Description value. ]
Yes...will have to be addressed in import.


Performance
  1. When exporting images: 1%=2m, 3%=3m, 5%=4m, 7%=5m, 27%=6m, 37%=7m, 47%=8m, 54%=9m (INDIs done), 100%<10m
    [ Presumably most of your images are linked to first few Individuals? ]
    Possibly, notably many of my images are cropped. The key matters are 1) its quick without images 2) its acceptable even when dealing with lots of images 3) run on a medium quality laptop.
  2. When exporting images but they already exist: 100%=70seconds
  3. When not exporting images: 100%=100seconds

Code: Select all

Record Counts...
    Individual Records:3163
    Family Records: 1065
    Note Records: 7
    Source Records: 9
    Repository Records: 24
    Submitter Records: 3
    Submission Records: 1
    Media Records: 990
    Place Records: 768
Use of Source Records
  1. Sorting using Omega works well.
  2. File Root looks OK [ linked to from Root Individual. ]
  3. Header Rec looks OK
  4. Named List looks OK [ linked to from every named record except REPO/SUBM/SUBN. ]
  5. Place Rec looks OK [ linked to from every matching PLACe field. ]
  6. Repository looks OK with Record ID appended to match Repository Record
  7. Submission looks OK with Record ID appended
  8. Submitter looks OK with Record ID appended
  9. Note that some of these are not linked to (List and Root are - no Fake person) therefore the tabs for Links, Notes and Media are disabled. To see details double click Source. Should we provide 1 link to a fake person (or the fake event of the root person).
    [ Most of Place Rec should have links from Place tag, but a bug had crept into 11 Apr 2015 Plugin, and now fixed in 13 Apr 2015 version. ]
    [ Cannot use the fake event of root person, as may not exist if no _ROOT in HEAD or if Removed entirely in Extra Options. ]
  10. Should there be a record above this group with a name like "DO NOT DELETE THE RECORDS BELOW".
    [ Good idea, now included in 13 Apr 2015 Plugin.
    What if this record links to all the others via its NOTE - SOUR? ]
    I can't see a way for FTM to allow a link from SOUR to a SOUR even via a NOTE - did I understand correctly?
    [ In Standard Gedcom & FH every local Note allows a Source Citation but maybe not in FTM: ]

    0 @S26@ SOUR
    1 TITL Ω Source Title
    1 NOTE Local Note
    2 SOUR @S12@
    [ 13 Apr 2015 Plugin links that warning Source Record and a new Note Record to all synthetic Source Records to see if either work in FTM. ]
Minor Tags
Not all tested at this stage. KB updated.

Mods 1-11
Although not tested in detail these appear ok. Let me know if there is something specific. I'll come back to this when time allows.

Initial observations
Media has numerous fields stored in Note - check them

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 14 Apr 2015 18:51

I've looked at the links (citations) to the omega sources using the latest plugin with the Source warning record.

The sources with links are:
File Root
Named List (when there are list members)
Place Rec (only Spouse Sealing Place works FAM, SLGS - no others)

The sources without links are:
Place Rec
Repository
Submission
Submitter

I can see the data that attempts to link a SOUR, NOTE, SOUR to each SOUR - nice thought but I cannot see any sign of this in or exported from FTM.

Further, I can see the data that attempt to link a 2 PLAC to a SOUR. Again, I cannot see 3 SOUR in or exported from FTM. Of course a 2 SOUR would work being a citation from the event rather than from the place of the event. Indeed the data for 1 SLGS has a 2 SOUR under the 2 PLAC - this works but is probably not what you intended.

I understand the intention of the Place data in Source records is to preserve such data for the import. I acknowledge the desire for links but unless I'm missing something all of them are unnecessary, indeed FH itself does not link to place records or lists. Should we consider keeping it simple and just preserving the data or is there another approach?

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 14 Apr 2015 19:23

Now that HEAD tags are all saved in a Source Record Ω Header Rec the Ω File Root and Ω Named List are optional. If included, then I think the Citation links may be useful, and I would like to keep them for consistency with exports to other products.

The Source Record Ω Place Rec Citations from LDS tags are a special case, because Gedcom does NOT allow PLACe field Citations here. The use of these LDS tags is rare, but again I would prefer to keep the Citations for consistency.

The Source Record Ω Place Rec Citations from PLACe fields are ignored by FTM, but again I would prefer to keep the Citations for consistency.
FH does not use Citations to link PLACe fields to _PLACe Records, but they are linked by Name.
The exported Citations fulfill a similar linking function, where the target product supports them.
(The Ω Named List Citations fulfill a similar role for named records, that are linked in FH.)

The other Source Records: Ω Header Rec, Ω Repository, Ω Submission & Ω Submitter have no Citation links.

The primary purpose of these synthetic Source Records is to preserve data for return to FH, so do they need any synthetic Citations? If without them it is more difficult to access their data, then maybe that is a benefit?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 14 Apr 2015 20:10

OK as long as you know what's happening and are happy...thanks.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 14 Apr 2015 20:21

OK, as long as you are happy that without any Citations these synthetic Source Records import to and export from FTM completely intact.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 14 Apr 2015 21:17

I think the use of Source records to hold the data works well.

As far as presentation is concerned it is a shame that Source records without a citation result in disabled Links, Notes and Media tabs. One can still double click the source to the record. This is the reason I manually linked my sources to a fake INDI.

We might get questions about this and that SLGS (and some others?) have links but the majority do not.

Functionally I believe it does the job and if the data can be imported back into FH even after FTM has ignored the links then the goal is achieved.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 15 Apr 2015 12:31

To minimise the risk of some synthetic Source Records having Citations and some not, I propose to change the default setting for Named Lists and File Root to Remove entirely. Then the only Source Records that might have a Citation are those Place Rec associated with a PLACe field subsidiary to an LDS tag: BAPL, CONL, ENDL SLGC & SLGS.

Talking about LDS tags such as SLGS prompted a look at their subsidiary tags, and revealed a few more that I suspect need moving to their local Note, i.e. 2 TEMP & 2 STAT & 2 FAMC.

What I have overlooked in the discussion about Source Citations is that FTM imports very few of all the possibilities.
As you have discovered, INDI/FAM whole record Citations need a synthetic Custom Event <whole record citation>.
Similarly, NOTE whole record Citations are not imported to FTM it seems.
Also it seems PLACe field Citations, and any local NOTE Citations are not imported. (Please confirm the latter)

So in those cases it is pointless copying OBJEct Media from Source Record to Citation.
I plan to simply move the SOUR tag and @link@ and any subsidiary tags to a local labelled Note for conservation.

Have you checked Source Notes?
These are a SOUR tag in exactly the same position as a Citation SOUR tag but instead of an @link@ its value is plain text.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Post Reply