* Ancestry, FTM, FH and workflow

Importing from another genealogy program? This is the place to ask. Questions about Exporting should go in the Exporting sub-forum of the General Usage forum.
avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 15 Apr 2015 19:37

Synthetic Sources: I think changing the default on ROOT is a good idea. As far as all defaults are concerned my thinking is to try to preserve all the data users might want on the basis few read the small print! I think documenting clearly that Named Lists can be excluded and the simple rule that only those specific flags get place citations is better. Of course, better still would be to find a way to make some kind of link to all Sources...but a simple solution is not jumping at me just at the moment - there must be a way (fake person or otherwise).

I suspect other tags will crop up later too and you have a tidy solution when they do...lets face it you've gone a lot further with the use of tags in notes than initially envisaged initially in a "demand driven" approach (perhaps the illusive "demand" is waiting for the import!).

Yes FTM is straightforward and somewhat singular in operation to fit with Ancestry, development constraints and a mass user base (that was being polite) whereas FH is far more mature and flexible. There are few neat tricks in FTM, that said it really isn't that bad, it presents well, is relatively simple, holds most of the data one might want and does the Ancestry link quite painlessly.

I believe the statements you made are correct from what I've seen and I'll check out the local note citation and source notes. The only citations I can see in FTM are against INDI and FAM events. I hope I've not missed anything in my travels.

Long ago I moved my data to FTM and my FH data is now way out of date - one day I want it back there! Moving to FTM early, I have missed out on some important plugin functions (e.g. data in notes) but luckily my data is quite basic even if my use of FH queries, web creation, etc. was not.

And of course my Ancestry matching and understanding of this feature has progressed considerably...importantly I still confirm enthusiastically the great value of matching against Ancestry records (and I've not bothered matching users trees yet). After some weeks of reviewing I still have 650 people with several records each to review...it takes time but its worth it. The matches hints are surprisingly accurate and frequently deliver more information and indeed new people as well as all the citations and their images. I also get a few photos of people! What I'm saying is the export plugin is absolutely worth using.

Of course, reviewing also means updating your data and by far the best way is using Ancestry features instead of copying and pasting back to FH. Trouble is...we need the FH import. And of course the FH import will also make FH readily accessible and highly attractive to existing Ancestry users. Now where am I going with all this he asks himself :)

In about 3500 hints time (coincidentally about the same time as a holiday from developing export plugins) it'll be time to get back to FH. I aim to press on with matching but when I need a break I'll continue listing the translations and processing to get back to FH....just in case anyone feels like a little light coding...

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 16 Apr 2015 10:16

I am considering a slight departure from our current approach as explained below.

FTM only supports Description, Date, Place & Note fields for Event/Attribute/LDS tags, plus linked Citations & Media.
We use Description to hold Event Address (1st line) or Attribute Value via Event/Attribute tag line.
The Date & Place are simply retained from FH DATE & PLAC tags.
That leaves the Note field, which is the only multi-line field, but only one instance of this field is preserved, so multiple local Notes and linked Note Records all collapse into the one Note field.

The various current labelled local Notes in this field are growing in number and complexity, and after the round trip back to FH may present an increasing challenge to any input translation Plugin.

So apart from retaining the supported FTM fields/tags, I am proposing to simply move all the other FH tag lines into the Note field after any existing note text, similar to the synthetic Sources for Header Rec, Repository, Submission & Submitter.
This would include subsidiary PLAC & ADDR data, AGE, CAUS, etc, etc, and automatically includes any tags we've overlooked.
It would sweep up multiple instances of NOTE, NOTE @link@, and subsidiary SOUR tags, etc.

I am sorely tempted to include absolutely all the tags for the whole Event/Attribute/LDS entry, which would make reconstruction after the round trip very much easier for the import Plugin.
All that would be left will be the Event/Attribute/LDS + Description, 2 DATE Date, 2 PLAC Place, 2 OBJE @link@, 2 SOUR @link@ & 3 OBJE @link@ tag lines, plus perhaps some Citation tags 3 PAGE Where in Source, 3 QUAY Assessment, 3 DATA, 4 DATE Date, 4 TEXT From Source, but not 3 NOTE.

A similar approach could be applied to all tags not supported by FTM at the Record level for all records: INDI, FAM, NOTE & OBJE.

What do you think Nick?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 16 Apr 2015 13:51

Here are the test results - will get to your latest message shortly.

In summary, Fact citations and Fact Notes work whereas citations and notes attached to Fact, Plac do not.

WHAT WENT IN
0 @I3525@ INDI
1 NAME Ian Gerald /Smith/
1 SEX M
1 BIRT
2 SOUR This is a local SOUR tag under 1 BIRT
2 NOTE This is a local NOTE tag under 1 BIRT
2 DATE 3 DEC 2010
2 PLAC Zdroje, Szczecin, Poland
3 SOUR This is a local SOUR tag under 1 BIRT, 2 PLAC
3 NOTE This is a local NOTE tag under 1 BIRT, 2 PLAC
3 SOUR @S218@
2 NOTE Place Details: Zdroje, Szczecin, Poland

WHAT DISPLAYED IN THE SOURCES FIELD (as expected as it's simply a citation)
This is a local SOUR tag under 1 BIRT

WHAT DISPLAYED IN THE NOTES FIELD (as expected as it's simply a Fact Note)
This is a local NOTE tag under 1 BIRT

Place Details: Zdroje, Szczecin, Poland

WHAT EXPORTED
1 NAME Ian Gerald /Smith/
1 SEX M
1 BIRT
2 DATE 03 DEC 2010
2 PLAC Zdroje, Szczecin, Poland
2 NOTE @N1889@
2 SOUR Details: This is a local SOUR tag under 1 BIRT
...more

0 @N1889@ NOTE
1 CONC This is a local NOTE tag under 1 BIRT
1 CONT
1 CONT Place Details: Zdroje, Szczecin, Poland
Last edited by Nick-V on 16 Apr 2015 17:33, edited 1 time in total.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 16 Apr 2015 13:56

Had a quick look at your idea...need to think a little but fully support anything that is simpler to code, explain, deal with the unexpected...

...in essence I take it that all FH data (for INDI, FAM indeed all appropriate record types) will go in a note and then, where FTM has an appropriate field, we'll duplicate the data to it.

If that's the idea I see no reason for fancy titles or tabs (which invite data input), we could simply copy all the raw record data...e.g. "level tag value". I see no issue with this at all.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 16 Apr 2015 17:19

Yes, that is exactly what I am proposing.

Thanks for the feedback on Citations & Notes.
What happens when there are multiple instances of Citation Notes and Source Citations?
e.g.
1 BIRT
2 SOUR citation note 1
2 SOUR citation note 2
2 SOUR @S2@
2 SOUR citation note 3
2 SOUR @S3@

One other area that has not been checked is Notes and Citations on the NAME tag.
e.g.
1 NAME John /Smith/
2 NOTE local note
2 NOTE @N1@
2 SOUR citation note
2 SOUR @S1@
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 16 Apr 2015 17:39

Mike...thinking further about the simpler way of preserving data...

...this very simple approach works well for users wishing to employ FTM for matches then return data to FH. However, the lack of pretty formatting compromises those wishing to simply move to FTM or use Ancestry for presentation purposes. Nevertheless its a good idea so how can both requirements be met?

Perhaps quite simply...option A uses level-tag-data, and option B does something a little fancier yet automatic:

step 1: (? no level) "tag & ": " & tab & data" or something similar? (potentially a level/tag combination could be replaced with nice text from a table achieving much as you have today) e.g. 2 PLAC = Place
step 2 (a little better): exclude certain unnecessary/duplicated tags

I'm sure the simplification comes mainly from simply copying every line to the note...making them a little more meaningful might be a straightforward bonus.

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 16 Apr 2015 18:01

I'll think about it, but the difficulties surround the tags that can occur at many different levels such as NOTE, SOUR, OBJE, DATE, etc, and the rather meaningless @link@ values?

One idea I have been contemplating is two export options:
  1. (FTM) FTM 2014 for a pretty export only.
  2. (ANC) FTM/Ancestry for an export to support returnig data to FH, which needs the level numbers on the tags.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 16 Apr 2015 18:30

A further test:

What went in:

Code: Select all

0 @I3525@ INDI
1 NAME Ian Gerald /Smith/
2 SOUR Source 1
2 NOTE Note 1
2 SOUR Source 2
2 NOTE Note 2
2 NOTE @N11@
2 SOUR @S218@
2 SOUR Source 3
2 NOTE Note 3
2 SOUR @S218@
2 SOUR Source 4
2 NOTE @N13@
2 NOTE Note 4
1 SEX M
1 BIRT
2 SOUR Source 1
2 NOTE Note 1
2 SOUR Source 2
2 NOTE Note 2
2 NOTE @N11@
2 SOUR @S218@
2 SOUR Source 3
2 NOTE Note 3
2 SOUR @S218@
2 SOUR Source 4
2 NOTE @N13@
2 NOTE Note 4
2 DATE 3 DEC 2010
2 PLAC Zdroje, Szczecin, Poland
3 SOUR @S218@
2 NOTE Place Details:	Zdroje, Szczecin, Poland
What displayed in the NAME Sources field:
5 source citations (1 link 4 explicit)

What displayed in the NAME Notes field:

Code: Select all

Note 1

Note 2

Note 3

Note 4

@N11@

@N13@

@N13@

The text of a note.....
Exactly the same story for the BIRT Sources and Notes field.

What came out:

Code: Select all

0 @I1@ INDI
1 NAME Ian Gerald /Amith/
2 NOTE @N1889@
2 SOUR Details: Source 1
2 SOUR Details: Source 2
2 SOUR @S180@
2 SOUR Details: Source 3
2 SOUR Details: Source 4
1 SEX M
1 BIRT
2 DATE 03 DEC 2010
2 PLAC Zdroje, Szczecin, Poland
2 NOTE @N1890@
2 SOUR Details: Source 1
2 SOUR Details: Source 2
2 SOUR @S180@
2 SOUR Details: Source 3
2 SOUR Details: Source 4
...more

0 @N1889@ NOTE
1 CONC Note 1
1 CONT
1 CONT Note 2
1 CONT
1 CONT Note 3
1 CONT
1 CONT Note 4
1 CONT
1 CONT @N11@
1 CONT
1 CONT @N13@
1 CONT
1 CONT @N13@
1 CONT
1 CONT The text of a note.....

0 @N1890@ NOTE
1 CONC Note 1
1 CONT
1 CONT Note 2
1 CONT
1 CONT Note 3
1 CONT
1 CONT Note 4
1 CONT
1 CONT Place Details:	Zdroje, Szczecin, Poland

0 @S180@ SOUR
1 AUTH Place Rec Id [290]
1 TITL Ω Place Rec: Zdroje, Szczecin, Poland
1 NOTE
2 CONC Source Type:	Place Details
2 CONT Text From Source:	Map Plot:
2 CONT Text From Source:	Latitude:    N53.383333
2 CONT Text From Source:	Longitude: E14.633333

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 16 Apr 2015 18:34

I think the two options is what I am suggesting as option A and B - pretty versus return trip. The only difference is the level/tags are replaced by pretty text using a translation table. The same processing everywhere else...just a final lookup on moving to the note. Actually the two options might not be necessary - why not always pretty...

...why not always translate and in the import translate back...what I am saying is do what you suggested - copy the record lines to the note and then simply translate them (and back again).

If you throw me a few tags or any concerns I'll try to throw back some examples of what I mean...where a table lookup fails use the tag.

Yes the links can go.

Example:

This list of tags in the note gets translated:

Code: Select all

0 @I1@ INDI
1 NAME Gerald /Smith/
2 _USED Given
2 NPFX Prefix
2 NICK Günther, Gerry
2 SURN Suname
2 NSFX Suffix
2 SOUR @S12@
3 PAGE Where in source
3 DATA
4 DATE 1 JAN 1900
4 TEXT Text from source
5 CONT more lots etc
3 QUAY 3
3 NOTE Note
4 CONT More note
1 SEX M
1 EVEN
2 TYPE Internment
2 DATE 1 JAN 1900
2 PLAC Internment Place
2 ADDR Internment Address
2 AGE 99y 6m 1d
2 OBJE @O1279@
3 _ASID 2
2 NOTE Internment Note
...more
Into this (very rough examples and perhaps the levels can go completely if we set up the table with unique translations):

Code: Select all

0 @I1@ INDI
1 Individual's Name: Gerald /Smith/
2 Given Name: Given
2 Name Prefix: Prefix
2 Nickname: Günther, Gerry
2 Surname: Suname
2 Name Suffix: Suffix
2 Source or citation: @S12@
3 Page in source: Where in source
3 Data:
4 Date: 1 JAN 1900
4 Text of citation: Text from source more lots etc
3 Quality of citation: 3
3 Note: Note: More note
1 Sex: M
1 Custom event:
2 Custom event name: Internment
2 Event Date: 1 JAN 1900
2 Event Place: Internment Place
2 Event Address: Internment Address
2 Event Age: 99y 6m 1d
2 Media Object: @O1279@
3 Media order: 2
2 Event Note: Internment Note
...more

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 21 Apr 2015 14:20

I have posted Plugin V1.7.A in the KB.

It uses the new idea of copying tags to Notes but with 'pretty' labels.
It applies this method to INDIvidual, FAMily, NOTE, and SOURce records (except the synthetic omega ones).
Details:
Each level digit becomes that number of spaces to indent the details according to the level.
Each tag translates to a unique Label followed by a colon (:) and a tab character or two.
Each link @RecId@ is displayed as [RecId] plus the name of the record.
INDIvidual/FAMily Names, Facts & LDS Ordinances each have their own local Note of 'pretty' tags.
Otherwise, the 'pretty' tags are in the record local Note.
SOURces & OBJEcts supported by FTM Names/Facts/LDS Ordinances are excluded from their Notes.

The order of the Extra Options tab options have been moved into better groups.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 21 Apr 2015 14:34

Good news and a lot of work...I hope it has simplified things, for the future also...I'll take a good look and get back...BTW I think "pretty tags" is a marketing concept worthy of copyright :)

Initial Observations:
  1. I note the improved grouping of extra options and indeed the change of defaults...I think I would default to keep lists not remove them.
  2. No errors on import
  3. The leading spaces hierarchy presents as expected, however the data does not align, the use of tabs/spaces may be awkward (see the pasted example below - you may need to edit this posting to see what I actually get). I'm uncertain if it is intended to step the data but it would present better if aligned vertically. Interestingly the Source Notes are consistent, the data has a hierarchy (not that I like it but it works properly). I see no hierarchy on media notes but this may be as expected...need to check.
  4. Alignment of SOUR note is strange...the character (perhaps CR) between lat and long lacks a LF perhaps as it presents as below (edited). Same thing with Bookmarks (shown below also).
  5. I note the use of (very) large headings in the note field, FTM appears to sort fact notes and compile them into one field on the Tree tab whereas individual fact notes are shown on the person tab.
  6. INDI notes (or fact notes) appear to contain cont records...I wonder if cont and conc's should be processed in the "read line" (and "write line") routine so they are invisible to the program logic and simply appear as they should.


Summary: I see this approach working just fine. We need to consider alignment of data and the way tabs and cr/lf operate. I need to go thru the data step by step to determine what is happening before I can advise...

INDI

Code: Select all

Record Id:	[I219]
 Gender:		 M
 Parents: 	 [F29] ...of Lewin Joseph Smith and Sophia SEISER OR LEIFER OR BERCHER
 Spouse:		 [F125] ...of Ludwig Smith
 Note:		 Went or died Berlin maybe 14 Aug 1836 or 1941
 Changed:		 
  Date:		  14 FEB 2015
   Time:		   02:23:36
SOUR (edited to show effect of no CRLF)

Code: Select all

Map Plot:
Latitude:    N51.5456033Longitude: W0.2331974
Source Type:	Place Details

INDI:	[I1]	Gerald SMITHINDI:	[I3525]	Ian Gerald SMITH
Last edited by Nick-V on 22 Apr 2015 07:59, edited 5 times in total.

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 21 Apr 2015 19:08

Source records for places (and named lists that seem to use TEXT):

Note that the notes for normal source records work OK although my preference is to indent the tag but not the data.

When there is a citation to a source (i.e. Named List, Spouse Sealing Place) then the tabs in the bottom panel for Links (showing person and fact), Notes (showing nothing but intended for Source Notes) and Media (showing nothing but intended for Source Media) are enabled. Further the fields in the right panel are enabled including Reference Note (which contains "Place Rec Id [P1245], Ω Place Rec: , Aberdeenshire, Scotland.").

Where there is no citation both the tabs and fields are disabled. Double clicking the Source pops up a record window showing tabs for Source (containing source record fields) and Media (containing media). On the Source tab the comments field presents like:

Presentation:
<blank line>
Map Plot:Latitude: N57.222645Longitude: W2.7778485
Source Type: Place Details

The data creating this is:
0 @S372@ SOUR
1 TITL Ω Place Rec: , Aberdeenshire, Scotland
1 AUTH Place Rec Id [P1245]
1 TEXT Map Plot:
2 CONT Latitude: N57.222645
2 CONT Longitude: W2.7778485
1 NOTE Source Type: Place Details

Questions:
Again, should we create citations to all sources somehow to enable these tabs and fields or are they acceptable as they are?
Should we place TEXT data into the NOTE instead to avoid the formatting issues (tested below) - also for named lists?
Are we happy with the use of Author for "Place Rec Id [P1245]" or use the note (see below)?

Alternative data (note two tabs on one - maybe make name longer):
0 @S372@ SOUR
1 TITL Ω Place Rec:{tab}, Aberdeenshire, Scotland
1 NOTE Source Type:{tab}Place Details
2 CONT Place Rec Id:{tab}[P1245]
2 CONT Latitude:{tab}{tab}N57.222645
2 CONT Longitude:{tab}W2.7778485

Presentation (perfect although not clear in the forum!):
Source Type: Place Details
Place Rec Id: [P1245]
Latitude: N57.222645
Longitude: W2.7778485

EDIT: Perhaps the indent should be SPACES(level-1), there is no need for 1 space on the top row being the title or fact.
Last edited by Nick-V on 22 Apr 2015 08:01, edited 1 time in total.

User avatar
jimlad68
Megastar
Posts: 911
Joined: 18 May 2014 21:01
Family Historian: V7
Location: Sheffield, Yorkshire, UK (but from Lancashire)
Contact:

Re: Ancestry, FTM, FH and workflow

Post by jimlad68 » 21 Apr 2015 21:15

I am following this thread every now and then, but can't say I understand much of it, but I do applaud what I think is the intent of portability of data, best of luck.
Jim Orrell - researching: see - but probably out of date https://gw.geneanet.org/jimlad68

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 21 Apr 2015 22:54

Thanks Jim...we're trying to get our valuable Family Historian info into Ancestry.com (via their Family Tree Maker software) so we can match against vital records etc...we're there now...next we want to get it back to FH again !

User avatar
jimlad68
Megastar
Posts: 911
Joined: 18 May 2014 21:01
Family Historian: V7
Location: Sheffield, Yorkshire, UK (but from Lancashire)
Contact:

Re: Ancestry, FTM, FH and workflow

Post by jimlad68 » 21 Apr 2015 23:30

Nick, nice to know, as Ancestry more or less give away FTM (perhaps that is why it is the most popular!), and many of us have a tree on Ancestry, so even if it is just a one way share to Ancestry it is nice to get more data there.
Jim Orrell - researching: see - but probably out of date https://gw.geneanet.org/jimlad68

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 22 Apr 2015 12:29

Nick, I've updated the Plugin in the KB dated 22 Apr 2015.

I think it fixes many of your criticisms.
The 'pretty tags' now only indent the tag and not the data.
The poor Text From Source format in synthetic Source Records was a bug that crept in.
However, the synthetic Source Records derived from Place Records now use the 'pretty tag' style for FTM. (It might be possible to extend that style elsewhere too, if thought desirable.)

I'm confused (again) about the Extra Options for Named Lists.
In the early days you consistently said you didn't use Named Lists and FTM didn't support them, so wanted them Removed entirely. Now I have set that default option, you say you want them retained in synthetic Source Records.

Remember that Named Lists (and File Root) are preserved for the round trip in the special synthetic Source Record for the Header, so do not need to exist separately.

The other reason for removing them (and the synthetic Source Record for File Root) is to eliminate all the Citations of synthetic Source Records (except for LDS Ordinace Places which could also easily be removed). Thus all the synthetic Source Record tabs are disabled to protect the details against accidental changes.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 22 Apr 2015 13:49

Mike, thanks...yes I don't use Named Lists, I am thinking about other users...sorry for confusion. I am keen that defaults preserve data and options allow users not to preserve data. I'm sorry I didn't quite understand fully that the necessary data for Named Lists survives the round trip anyway in the header so all is OK.

Yes disabled Source Records (unless you double click them - so perhaps "Awkward" rather than "Disabled") is a good argument for not having citations...agreed.

Thanks for spinning round another version with new improved "pretty tags", etc. Alignment looks a lot better. I need to look further to find out why the following issues exist. Comments:

The indentation could SPACES(level-1) instead of SPACES(level) - the indent at level 1 is unnecessary.

Person Note
Parents: (too little space) data (Spouse works fine)
Cont records (blank and with data) - need to think about these a little, didn't notice CONCs, can we make this work without CONT...if not then fine.

Fact Note
Address: (too little space) data

Object Note
Blank line at start

Source Note
No issues on alignment
HEAD, REPO, SUBM and SUBN have no indentation (for info)

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 23 Apr 2015 15:02

Updated Plugin in KB dated 23 Apr 2015.

I think it fixes most of the problems.
Handles long Note, Address, Text fields with CONC/CONT as requested.
No Source Citations for any Place fields to synthetic Source Records.

I would like to keep the space indents to match tag level because it is easier to describe, understand & implement. Also we might one day need to include a 'pretty tag' for a record header at level 0.

I've looked at the Object Note blank line, and it is awkward to avoid, because every Link/Note in original GEDCOM is a minimum of a blank line, but can be any multi-line long text Note. It also needs to be retained for round trip consistency. In the longer term it might be possible to use 'pretty tags' in the Media Object records, but then every one would start with at least one blank Note: line.

HEAD, REPO, SUBM and SUBN synthetic Source Records are plain copies of tags, not 'pretty tags', because nobody needs to look at them, as they are only for round trip survival, but I am open to persuasion.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 23 Apr 2015 18:21

That all sounds great. Thanks. I accept your reasoning for the space indents etc. I see no compelling argument for pretty tags in those other records...the only small issue is changing later as the import would have to decode both approaches.

I'll grab the new plugin and have a good rumage around over the next little while to see if there is anything picky I can find...Is there anything I can do (screen grabs, export file, etc.) to provide you sight of the outcome of all this so you can see for yourself if there is anything else we should consider?

I wonder if we can enlist one or two helpers to throw their data at this and provide fresh feedback.

My own data concerns me as mentioned previously...it's been in FTM (and not in FH!) for a while and I'm wondering how I might merge in the new functions of the plugin so it can return to FH one day. I'm only matching...but this causes data to change, so simply adding the pretty tags from a few weeks ago will not solve the problem...it's not going to be easy!

Observations below (gradually):
  1. Person Note: Attribute:{not enough tab}value
  2. Fact Note for NCHI: Children:{not enough tab}value
    [Mike says: I wondered if there might be tab problems, because it depends on screen font size, etc, which vary on different PC, so we might have to abandon tabs and either use one space or put value on next line with same indent as tag label.] [Nick replies: I will try to establish a little more what is happening with tabs...I understand the point, I cannot run FTM on another PC but I can change resolution...first to check and edit the GEDCOM file... Mike answers: I suspect changing default font is more relevant than changing resolution.]
  3. CONT has gone but ":" remains. Further data is tabbed (after therethe missing pretty tag) but of course its not a hanging tab so subsequent lines do not indent. Ideally notes would have not tag, colon or indent...just flow as normal.
    [Mike says: Cannot eliminate indent, tag label & colon because Notes occur at many different levels and multiple instances at the same level, so without the indent & tag label they cannot be faithfully recreated after round trip. Also the same strategy must work for other long text fields such as Address and Text From Source that have multiple CONC/CONT lines. If tag label is removed we cannot differentiate Note from Address from Text From Source.] [Nick replies: Yes understood, indeed the current approach (except the ":") works well in the Source Note - even when there are 2 notes (the second gets a pretty tag). Mike answers: I felt the ":" implied a continuation of the Note. Removing it will not alter the flow of the text. Also without a ":" there is no 'pretty tag' at all for the round trip input plugin to detect where a CONT belongs, but maybe the tab is enough.]
Last edited by Nick-V on 23 Apr 2015 22:37, edited 3 times in total.

User avatar
BillH
Megastar
Posts: 2184
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Ancestry, FTM, FH and workflow

Post by BillH » 23 Apr 2015 19:49

Nick,

I have been following along a bit with the discussions between you and Mike, but I don't really understand all the details. I was just wondering. Does this approach require FTM or can you go directly from FH to Ancestry and back to FH without using FTM? If it does require using FTM, I guess I would wonder why. What does FTM buy you?

Thanks,
Bill

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 23 Apr 2015 20:47

Hi Bill

Thanks for trying to follow all this stuff...I'm pleased to hear there is some potential interest out there despite how boring and technical this may sound at times!

I hope I can answer your question clearly...why do we need FTM, why not simply link FH directly with Ancestry? Reasons include:
  1. We do not have any technical information about how Ancestry's web to PC synchronisation works - it will be complex for sure.
  2. If we did manage to find out, we would have no prior warning or control therefore when the spec changes our approach would suddenly become unusable - further there would be an on-going support burden.
  3. We cannot link FH with Ancestry simply by sending and receiving GEDCOM files - use of GEDCOMs varies between software authors (despite it being a "Standard") and again, when they change their approach we would not know!
  4. GEDCOMs don't hold the images so uploading/downloading and connecting images to the GEDCOM is not easy (or indeed possible) to automate.
  5. We wish to preserve Ancestry hints/matches. I have worked thru thousands of hints with many still to go...I would have to check them all again if I uploaded a new GEDCOM.
So...we decided to use FTM as the middle man...it costs £20 which is nothing in comparison with the costs of your time researching, Ancestry subscription fees, development costs etc.
  1. FTM will always synchronise with Ancestry for both GEDCOM and images - they have to commit to that.
  2. FTM also provides a local GEDCOM and images which, after some work (Mike's plugin), can work alongside FH.
Excuse the quickly bashed out answer - I hope it supports the decision...

...BTW, this is only half the story...you can now (soon) get your data to Ancestry and use their features for matching and updating...however we do not yet have an acceptable path back to FH although sensible decisions have been taken to make that development more feasible...

User avatar
BillH
Megastar
Posts: 2184
Joined: 31 May 2010 03:40
Family Historian: V7
Location: Washington State, USA

Re: Ancestry, FTM, FH and workflow

Post by BillH » 23 Apr 2015 21:42

Hi Nick,

I guess I was thinking maybe we could just send and receive GEDCOM files but I see from your explanation why that won't work. If you are able to get the round trip to work, I'd definitely be interested in using the final product.

I'd offer to help at this point, but I don't currently have either FTM or a tree on Ancestry. I've been holding off with the tree on Ancestry because of the very problems you are trying to overcome. I love FH and don't want to switch to FTM and I really don't want two trees that I have to manually keep in sync in FH and on Ancestry. If you get the round trip working, I'd be happy to buy FTM and do some testing at that point if it would help.

Thanks,
Bill

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 23 Apr 2015 21:51

Thanks Bill

I'm hoping there will be a few of us interested in the finished round trip. It must be true that FH is better AND we need Ancestry !

Thanks for your offer of help...keep tuned and we can work together when the right time comes !

avatar
Nick-V
Superstar
Posts: 268
Joined: 18 Nov 2009 17:50
Family Historian: V6
Location: London, England

Re: Ancestry, FTM, FH and workflow

Post by Nick-V » 23 Apr 2015 22:51

Mike...I'm looking at tabs for NCHI Children value and custom attribute value. Obviously I can only comment on my setup...(res, fonts etc). FTM appears to have a choice of standard or large font only.

In the FTM note field, when I delete any spaces and tabs between the pretty tag and value then put two tabs everything aligns well (using FTM's chosen font). This is true even with "Parents:" which works fine with two tabs when I replace the three unknown (tab/space) characters (which also work). Then we hit Permanent Ref: which is long...so only one tab is required...

When I go back to the gedcom both of these examples appear to align (using notepad and its selected font). All of the tags I looked at had 2 tabs...however, custom attrib has only one. If it had two it would not align in notepad but would in FTM. Children also appears to align in Notepad but may have 1 space and 1 tab.

I'm not sure how you've approached this (use of tabs and spaces) so I can't recommend an answer but hope that info helps...are there any other particular tags I might look at more carefully?

That said...you might be right that the issue is less controllable...when I make the whole note bold it really screws up alignment, items with longer tags tab further to the right. Also when I change to large fonts. It takes 12 spaces to align to a tab or 16 spaces to a bold tab ! We can't simply replace characters with spaces as it is not a fixed pitch font...and tabs do a great job of alignment so would be sad to lose them.

Let's start some silly options and see what happens:
  1. one space only - data would not align nicely as with tab

    Code: Select all

    Record Id: [I1]
     Gender: M
     Parents: [F8] ...of Julius SMITH and Gertrud SMITH
     Spouse: [F3] ...of Gerald SMITH and Gloria Miriam SMITH
     Permanent Ref: 36533819
     Changed:	
      Date: 4 APR 2015
       Time: 16:52:41
    	
  2. data on separate line - would take a lot of space but tabbed alignment could be achieved

    Code: Select all

    Record Id:
    		[I1]
     Gender:
    		M
     Parents: 
    		[F8] ...of Julius SMITH and Gertrud SMITH
     Spouse: 
    		[F3] ...of Gerald SMITH and Gloria Miriam SMITH
     Permanent Ref: 
    		36533819
     Changed:	
      Date: 
    		4 APR 2015
       Time: 
    		16:52:41
    
  3. make indent and pretty tag of a more consistent length - still a small risk with strange fonts, indeed the mess below works in FTM!

    Code: Select all

    Rec ID:		[I1]
     Gender:		M
     Parents:		[F8] ...of Julius SMITH and Gertrud SMITH
     Spouse:		[F3] ...of Gerald SMITH and Gloria Miriam SMITH
     PermRef:		36533819
     Changed:
      Date:		4 APR 2015
       Time:		16:52:41

User avatar
tatewise
Megastar
Posts: 27088
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Ancestry, FTM, FH and workflow

Post by tatewise » 24 Apr 2015 08:45

Nick, as I suspected. The same problem affects Notepad, FH Notes, etc. Without definable tab stops, the layout will almost always be at the mercy of the font selection.

Regarding the options you have considered:
  1. One space works, but as you say is not very tidy.
  2. Data on a separate line works well, and is my preferred solution. The vertical display space is almost doubled, but file storage space is similar to other options.
  3. Making every indent+pretty tag label a consistent length is an option that I considered and rejected.
    Some tags have very similar labels that need to be different enough to differentiate the tags, resulting in some very cryptic labels if they must be short.
    Also the indent can exceed 6 spaces in some cases, so any affected label+tab must work for every indent from 1 space to say 9 spaces. This is one reason why the current format can behave inconsistently. Some of the most common examples are where a field with a NOTE has a SOURce and that has sub-tags that may also have SOURces.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Post Reply