* Ancestry, FTM, FH and workflow ~ Media

[tatewise]

This thread continues from Ancestry, FTM, FH and workflow (12388) but focussed on Media issues associated with exporting BOTH full and part frame images that can complete the round trip from FH to FTM back to FH.

Nick asked: Is it relatively straight forward for the BOTH option to create the multiple links to the full images?

Mike replied: As you know, the BOTH option has multiple links to full images for local Media Objects already. There is no problem with BOTH option multiple links to Media Records linked to full images, because they already exist. The more complex job is linking to newly created Media Records linked to the new part images, but I think it is feasible.

Nick said: Mike, my head is spinning (tired)! I don't understand "multiple links to full images for local Media Objects". Don't worry, I'll create a BOTH file and look again at what's there. When I last looked I though the part images are attached to the INDI (LMOs?) and the full image are OBJE record with no links to anything yet (which was a bit odd!).

Yes I understand creating the new links to the full image (the 0 prefix record) might be "interesting"...I'm not quite there yet!

[tatewise]

Nick then replied:

Here is some BOTH data using my terminology (cross fingers):

' Below is the full image as a media record with a "0" prefix. There are no links to @O210@ at all so it can only be seen in a "media library", not against an individual or family.

0 @O210@ OBJE
1 FORM jpeg
1 TITL John Smith 1948
1 FILE C:\Users\...\Family Tree Maker\Test2 Media\0O210 0020.jpg
1 NOTE Keywords: picture keywords
2 CONT Media Date: 1948
2 CONT Picture Note: picture note
2 CONT

' Below is a part image taken from that full image but as an object attached to the individual - no links are required.
0 INDI
1 NAME bla bla
1 OBJE
2 FORM jpeg
2 TITL bla bla 1948
2 FILE C:\Users\...\Family Tree Maker\Test2 Media\1O210 0020.jpg
2 NOTE

So for clarity:
1) both objects import to FTM so no changes necessary however...
2) you mentioned all images might be media records in future (although it doesn't matter for this purpose)
3) INDI links "1 OBJE @O210@" need to be inserted. Several individuals might point to the same media record.
4) INDI tag "1 _PHOTO @O210@" would tell FTM the profile picture to use. Only to be created if it does not already exist.

[tatewise]

Actually, the current BOTH option uses Local Media Objects (LMO) for both the part frame and full images:
0 INDI
1 NAME bla bla
1 OBJE
2 FORM jpeg
2 TITL bla bla 1948
2 FILE C:\Users\...\Family Tree Maker\Test2 Media\1O210 0020.jpg
2 NOTE text bla bla
2 REFN custom id
1 OBJE
2 FORM jpeg
2 TITL bla bla 1948
2 FILE C:\Users\...\Family Tree Maker\Test2 Media\0O210 0020.jpg
2 NOTE text bla bla
2 REFN custom id

It also unintentionally kept the Media Records for the full images. They should have been removed.

In response to your points:

1) You say both objects import to FTM, but does that include all the tags & values for FORM, TITL, NOTE, and REFN?
NOTE should allow both text with CONTinuation & CONCatenation tags, and also NOTE @N1@ links to Note Records. This could be crucial to support point 5) below.
Are multiple instances of NOTE tags supported?

2) An alternative BOTH style option could produce a Media Record for every part frame and full image. Then all Media would use the 1 OBJE @O210@ format, but with new Record Id for the new part frame records. I think this method is crucial for your purpose, see 4) below.

3) You are proposing a hybrid of the above with 1 OBJE @O210@ links to full frame Media Records, and with LMO for any part frame images, but this has problems, see 4) below.

4) Using 1 _PHOTO @O210@ is only possible if the preferred image is in a Media Record, BUT if part frame images have been used, the preferred image is usually one of them, so part frame images must be in Media Records, not LMO.

5) I suspect users will want the Media to be restored to FH style part frames on return from FTM, so the custom _AREA, _DATE, _KEYS, _NOTE, etc, tags & values need to survive the FH/FTM journey somehow, perhaps in NOTE text. Also the filenames all need their prefix removed.

6) Media sub-folder structure has not yet been mentioned. I suspect users will want their carefully crafted folder structure retained throughout. If the Export Gedcom plugin were to support it, would that be possible in Ancestry/FTM?

7) You have promoted the use of absolute file paths because they survive the FH/FTM journey, but it is essential they are restored to relative paths on return to FH. Otherwise, FH Projects cannot be synchronised across PC, and cannot safely migrate from PC to PC, without breaking the links. In the case of PC failure, it is important that Project backups use relative paths to allow them to be easily restored on another PC.

8) Most of this discussion has focussed on Individual (& Family) Media that in FH are attached to the Media tab or directly to Facts. But the main purpose of the exercise is to capture Ancestry/FTM hints and associated Source/Citation Media. Have you any thoughts on how such Source/Citation Media survives repeated round trips?

[Nick-V]

Some quick feedback:

1) I didn't notice the plugin exported full images twice, once as LMOs and once accidentally as MRs. My initial answer is that to say what FTM appears to do. It store Caption, Date, Categories, Description, Private flag and a note record and exports:

0 @M355@ OBJE
1 FILE C:\filename.jpg
2 TITL Edward Stevens 1940
1 _PRIV
1 NOTE @N355@

' A bunch of rather unhelpful blank note records and this one:
0 @N355@ NOTE
1 CONC This is a note. This is a note. This is a note. This is a note. This
1 CONC is a note. This is a note. This is a note. This is a note. This is a
1 CONC note. This is a note.

I've mentioned previously that FH Date needs to be put (manually) into the title to be retained.

2, 3 and 4) Agreed. I believe that is what I meant. All INDIs , FAM, SOUR etc would use the link? So no LMOs at all.

5) Removal of prefixes on import will resolve the potential problem of prefixing multiple time on every export.- good, solved. I would regard preserving the other data including crop information as a stage two bonus. Regarding all of this a stage one would provide the facility quickly but with compromises including lost data. Stage two(s) would seek to preserve more or all data - could be delivered incrementally according to user priority.

6) I had no idea FH users kept their media in separate folders. No, FTM has just one folder for all media and its name is fixed. We need to consider overwriting of media exported to FTM previously (filename(1).jpg etc.). Again a stage 2.

7) Yes, I regarded this as a simple rename of paths.

8) Yes, a bit. As mentioned previously FTM events have potentially three bits of relevant info: 1) link to Source (single 1911 census for everyone), 2) an object for the census for just that family in 1911 and 3) a PAGE record with text. The source record would need duplicating on import to FH and re-merged on export for round trip.

0 @I19@ INDI
1 NAME Matilda /Ridgway/
2 SOUR @S10@
3 PAGE Class: RG14; Piece: 10332; Schedule Number: 36
3 OBJE @M2547@
etc.

[tatewise]

My responses:

1a) Please check how FTM imports FH Media Records with 1 REFN Custom Id, and multiple 1 NOTE tag instances (either text or links to Note Records).
e.g.
0 @O210@ OBJE
1 TITL Edward Stevens
1 FORM jpg
1 FILE C:\filename.jpg
1 REFN custom id
1 NOTE text ba bla 1
1 NOTE text ba bla 2
1 NOTE @N1@
1 NOTE @N2@

1b) The returned FTM Media Record has Title on invalid level 2 TITL and has lost FORMat tag. Is it coincidental or deliberate that Media Id and Note Id are the same (355)? Presumably you want _PRIV tag to survive roundtrips?
i.e.
0 @M355@ OBJE
1 FILE C:\filename.jpg
2 TITL Edward Stevens
1 _PRIV
1 NOTE @N355@

The solution to all tag data surviving the roundtrips is to save it in a NOTE somehow, but the options depend on answer to 1a) above.

2, 3 and 4) Yes, all Media Records and no LMO for this FTM strategy.

5) Cannot remove prefix without reinstating the part frame crop _AREA, because all the full & part images would then have same filename! Maybe initially keep part image prefix, and change Export Gedcom plugin to never add prefix to full images. Then after first cycle all images are full images, so they never change filename again. But users may add new part _AREA frames, and plugin must ensure every prefix+filename is unique! Not solved!

6) Yes, Media folder structure is a big issues for some users. Yet another bit of data to survive the roundtrip.

7) OK, but that is not mentioned anywhere in the KB advice.

8) I get that Census Sources/Media need complex handling, but what about all other life event Sources, Birth/Marriage/Death Certificates, Electoral Rolls, etc?

[Nick-V]

A quick reply on the easy bits. Further research later.

1a) Later

1b) Media privacy needs further consideration - we try to preserve what we can (either now on in a later version) to satisfy the owning software, or we try to convert FTM security flags to FM (if exists) satisfying both or we ignore this tag in favour of other things and users do without. For me, do the basics and think about _PRIV later.

234) Good

5) Same filename oops...yes more thinking..

6) in version 2 or whatever.

7) Not in KB(2) as export already does it. Already mentioned in KB(6).

8) I used Census as an example FTM probably doesn't care what type of vital record, more research required but I believe and vital record has the same links...we just need to be clear which ones WE wish to split and remerge...census is an obvious example. It could be that all Birth Certificates are regarded as a single source...more later.

[tatewise]

1a) Please at least try the Media NOTE tag checks, because that could offer a simple solution to roundtrip data survival. It is also needed for me to update the Export Gedcom plugin for Media Records.

7) Sorry ~ missed the KB(6) advice ~ I might add an explanation why.

The amount of feedback you have given on FTM Gedcom already vastly exceeds anything anyone has provided before, and will also benefit users who simply want a one-way export to FTM.

[Nick-V]

WRITTEN AGAIN AS TESTING WAS WRONG - working at it...

I had to create links from INDI to the OBJE records as these records are currently orphaned output of the BOTH option...hopefully that will get proper results. I also added your test tags. Note RFN on the INDI record did not re-export from FTH.

On import FTM reported import errors for Line 46662: error 4 : Unsupported or invalid tag: @N11@. Line ignored. In FTM the OBJE appeared against the INDI with a TITL but no other data or notes. These notes were not output.

See below for what went in and what came out.

Please note FTM allows formatting in notes field (Bold etc.) Also note that on fact notes the addresses and other data put from the export DO appear correctly.

Looking at the Export, there is no sign of the Note Records in the file. However, I did note that FTM exports loads of blank note records - it appears to export a note record for each INDI even if empty !

There was no difference when I used an M record (correctly linked to the INDI).

When editing all OBJE fields the only additional tag I got was 1 _PRIV.

0 @O210@ OBJE
1 FORM jpeg
1 TITL John Smith 1948
1 FILE C:\Users\......\Documents\Family Tree Maker\Test2 Media\0O210 0020.jpg
1 REFN 12345678
1 RIN 87654321
1 NOTE First Note
1 NOTE Keywords: picture keywords
2 CONT Media Date: 1948
2 CONT Picture Note: picture note
1 @N11@
1 @N13@
1 @N15@

0 @M148@ OBJE
1 FILE C:\Users\......\Documents\Family Tree Maker\Test2 Media\0O210 0020.jpg
2 TITL John Smith 1948

[tatewise]

Also try without the 1 RIN tag &/or without the 1 REFN tag as FTM might ignore everything afterwards. Alternatively, put those tags after all the 1 NOTE tags, because the order of tags on the same level should not matter.

The linked Note Record line should be 1 NOTE @N11@

I presume there were actual Note Records N11, N13, N15 in the Gedcom?

With the M123 Id try using Note Record with same Id number N123.

[Nick-V]

Too tired for this now...

INTO FTM

added to INDI
1 OBJE @M210@

0 @M210@ OBJE
1 FORM jpeg
1 TITL Gerry Smith 1948
1 FILE C:\Users\....\Documents\Family Tree Maker\Test2 Media\0O210 0020.jpg
1 NOTE First Note
1 NOTE Keywords: picture keywords
2 CONT Media Date: 1948
2 CONT Picture Note: picture note
1 NOTE @N210@
1 NOTE @N13@
1 NOTE @N15@
1 REFN 12345678
1 RIN 87654321

0 @N210@ NOTE TRANSLATIONS:
1 CONT
1 CONT Jacki Lewis.

ON IMPORT
No errors - note links correct

IN FTM
No different

OUT OF FTM
No different

I think it safe to assume FTM was designed with 1) Ancestry and 2) its own data needs in mind...there is no attempt to do much with import and export except the minimum necessary and certainly no intent to pass thru data.

[tatewise]

Earlier today on Fri Mar 13, 2015 2:15 pm you illustrated that FTM does export a Media Record with a linked Note Record containing note text.
But where does that text come from? Was it local to the Media Record or in a Note Record?

In the last posting you say that the Note Record imports correctly to FTM, so why does it not export?

It suggests that if an FTM exported Gedcom is imported back into FTM then data such as all Media Notes get discarded. Please confirm.

If Notes cannot survive the round-trip then they cannot be used to hold the data we want to retain, and it seems to me that the strategy is flawed before we've begun.

What about standalone Note Records, do they import and export reliably?

[Nick-V]

As a reminder to myself, you are seeking a way to round-trip note records to preserve data that might be used to re-merge into FH or indeed for FH data that FTM doesn't have a place for. I'll be coming back to use of notes in detail when I start working on exporting from FTM.

[tatewise]

Yes correct, I am looking for ways to preserved both FH & FTM custom tags & unsupported standard tags & associated data over the round trip.
Either Note Records or local Notes seem the most promising candidates, and maybe a separate thread is needed just to examine those options.
We know that FH handles & preserves them well, but what about FTM?

I wonder if FTM is being awkward with Media linked Notes (both Note Records and local Notes).
Maybe on import FTM only recognises the first (or perhaps the last) 1 NOTE tag of either flavour.
Maybe FTM imports local 1 NOTE text but exports it using 1 NOTE @N123@ with the text in the Note Record?

If all else fails, and assuming FTM will import and export any Note Record intact even if not linked to anything, then that may provide a vehicle for preserving round trip data. It would need a unique 'key' to link each each associated record with its data by somehow using record names/titles/etc to form the 'keys'.