* A couple Export Gedcom File questions

[rfield]

First post and newbie FH user.

I've been a long time TNG user for my web presence and have had challenges with converting exported gedcoms to be compatible, eg. media type, default photo, living, etc.

I was tired of running through the scripts and decided to give FH a try. I've used many programs for the last 30 years and I have to say I'm enjoying FH. I learn something new everyday and I'm really enjoying the program. I'm looking to make FH my new main program.

I then discovered this plugin. What could be better! So I've started playing and testing and hopefully this is just something I'm doing wrong but I've noticed the following.

I've exported a gedcom using ver. 1.9 and have noticed that I have a few orphaned objects. The following is an example fro a gedcom exported for TNG.

Code: Select all

1 OBJE
2 FORM jpg
2 TITL Sarah Ron and Katie
2 FILE C:\Users\Ron\Desktop\tng\1O3 cousins.jpg
2 NOTE 
3 CONT Picture Note:	Sarah Warwick, Ron Field, and Katie Bailey. Taken in England.
3 CONT Media Date:	1998
3 CONT Keywords:	Picture
1 OBJE @O4@
1 OBJE @O5@

The Media pointers @O4@ & @O5@ are orphaned and there's no target record for them in the exported file although the original gedcom does have them. One object is a PDF and the other is just a regular photo no frame(s). TNG creates a blank object when imported.

One other question I have, is the default photo for the individual is not correct on import. I have a few hundred (500) individuals with photos which gets to be a big job. I've noticed the plugin code supports setting the first individual photo with a '_PHOTO' tag for TMG. Do you think it would be possible to do the same type of thing and have a '_PRIM Y' tag added to the first individual image like this or maybe have it as an option? TNG recognizes this and will set them as the default individual photo.

Code: Select all

1 OBJE
2 _PRIM Y
2 FORM jpg
2 TITL Ron Field
2 FILE C:\Users\Ron\Desktop\tng\0O1 Ron Field HD.jpg
2 NOTE Media Date:	3 AUG 2013
3 CONT Keywords:	Picture

I had previously scripted gedcom changes to support the media types "_TYPE" tag for importing to TNG (PHOTO, DOCUM, HEADS, SOUND, VIDEO) so the media is sorted on import. I'm still playing with using a single media folder and for now I'm just using keywords to search and add to the different collection types.

I have to say that I have used RM7, FTM2014, Legacy 8, (sometimes all at the same time) and I'm finding that the things I found each program to be lacking I'm finding in FH and then some. I still have over 7,000 UDFs to clean up, Sources to re-do but I'm pretty much done cleaning up places (for the most part).

One final question, just to be sure, when you perform the export and specify the target folder not all media is to be exported to this target folder, correct? Just the ones with frames and the full version if that option is selected. I've noticed that headstones appear in the folder too and a few others. Possible a side effect of the UDFs?

Thank you for the great work!

Ron Field

[tatewise]

Welcome to the FHUG Ron. Wow! You have been busy!

As a newcomer you may find how_to:key_features_for_newcomers|> Key Features for Newcomers refers to topics you have not yet considered.

Sorry about the orphaned Media objects. That should not happen.
Could you please post the lines near the 1 OBJE @O4/5@ links and the entire 0 @O4/5@ OBJE records from your FH Project Gedcom, so I can investigate the problem.
Also confirm the Plugin Multimedia conversion option chosen, and that all other options use the default settings.

Yes, FTM uses _PHOTO tag links to OBJE Media Records (but not Local Media OBJEcts).
For TNG it should be possible to insert the _PRIM Y tag for the first Individual photo.
Does that only apply to Local Media OBJEcts as per your sample code, or does it also apply to links to Media Records?
e.g.
1 OBJE @O99@
2 _PRIM Y

Would putting FH Keywords into the TNG _TYPE tag be of any use?
If so then please post some sample code.

Have you found the how_to:handling_unrecognised_data_fields|> Handling Unrecognised Data Fields advice?

Your assumption is incorrect. All the necessary Media files are copied to the chosen Export folder whether part frames are involved or not. See the plugins:help:export_gedcom_file:the_faq|> Export Gedcom File ~ Frequently Asked Questions.

[rfield]

Hi Mike,

Thanks for the quick reply!

Here's the area that has the media links from the project.

Code: Select all

1 _ATTR working on classic Mustangs
2 TYPE Hobbies
2 DATE 2012
2 PLAC Winnipeg, Manitoba, Canada
2 OBJE @O7@
2 OBJE @O8@
2 OBJE @O9@
1 FAMC @F2223@
1 FAMS @F1@
1 FAMS @F2@
1 OBJE @O1@
1 OBJE @O2@
1 OBJE @O3@
2 _ASID 1
1 OBJE @O4@
1 OBJE @O5@
1 _FLGS
2 __LIVING Living
1 CHAN
2 DATE 29 JUN 2015
3 TIME 18:40:50

And the OBJect records for @O4@ & @O5@

Code: Select all

0 @O4@ OBJE
1 FORM jpg
1 TITL Ron Field
1 _FILE Media\M2 Field Ron.jpg
1 _KEYS Picture
....
0 @O5@ OBJE
1 FORM pdf
1 TITL Y-DNA Results for Ron Field
1 _FILE Media\Y-DNA Results Field Ron.pdf.pdf

The media conversion mode was set to (ALL~LMO).
I also had the "Named list entry" setting set to 'remove', everything else was set to the default.

You're correct with your example for linked objects I'll include examples of both Local and Linked.

-Linked-

Code: Select all

1 OBJE @M1@
2 _PRIM Y
1 OBJE @M2@
1 OBJE @M3@
1 OBJE @M4@

-Local- (2 _PRIM Y also works immediatley after the '1 OBJE' tag)

Code: Select all

1 OBJE
2 FILE C:\Users\Ron\Documents\RM_Media\photos\Field Ron HD.jpg
2 FORM jpg
2 TITL Ron Field
2 _TYPE PHOTO
2 _PRIM Y

Keywords and _TYPE was the first thing I thought of too.
There are two ways to setup the media object types.

One is to use directories, a separate directory for each type, eg. Photos, Media, Headstones, Documents.
I'm not sure how well this would work if everything is put into a single folder on export. This would also require a bit of work is you had to convert to using folders in FH just for TNG.

The second way is to have the _TYPE set for the OBJ. This can be used with linked records and should come immediately after the '0 @O456@ OBJE'.

Examples of linked and local.

-Linked-

Code: Select all

0 @M1@ OBJE
1 _TYPE PHOTO
1 FILE C:\Users\Ron\Documents\Family Tree Maker\Our Family Tree 2012 Media\p_FBProfilepic.jpg
1 FORM JPG
1 TITL Ron Field

-Local-

Code: Select all

1 OBJE
2 FILE C:\Users\Ron\Documents\RM_Media\photos\Field Ron HD.jpg
2 FORM jpg
2 TITL Ron Field
2 _TYPE PHOTO
2 _PRIM Y

I'm not sure of the best way to determine the Keyword to be used for the _TYPE.
A couple thoughts, first keyword in the keyword list or maybe MAP it in the plugin?
For example in the plugin if you specify keyword mapping, Photo or Picture -> _TYPE PHOTO, Document -> _TYPE DOCUM, Video -> _TYPE VIDEO etc.
The mapping may work since it would give you the choice of which keyword to use??

Thank you for the links, I had seen a couple of them before. The UDF one I saw but had concerns of loosing data. Today I did a backup and cleaned them out (saving the Log file for review). Looks good now!

Thank you for taking a look Mike.

Ron

[tatewise]

Hi Ron,

Are OBJEct records @O4@ and @O5@ the only ones without CHAN, DATE, TIME tags at the end?
If so, then that may be the cause and should be fixable.

Should be able to add _PRIM Y immediately after the appropriate OBJE tags of both flavours.

Are the _TYPE codes PHOTO, DOCUM & VIDEO standard TNG codes or your personal custom codes?

I am not keen on mapping, as that would involve another user dialogue, especially since none of the FH Keywords are fixed but entirely user customisable.

I would prefer a rule such as the first entirely upper-case Keyword is used as the _TYPE code.
In other words some way of choosing a Keyword to use as the _TYPE code.
Note that there is a Plugin to mange Keywords, so that is where the 'mapping' could be applied globally, and use the same Keywords/codes in both FH and TNG.

[rfield]

I'll check for other Objects and if they have CHAN, DATE, TIME or not and will post.

The supported _TYPEs below I found in documentation for a FTM to TMG converter.
------
TNG will take the first 5 letters of the following types. So DOCUMENTS etc. would work.

PHOTOs
HISTOries
BIOGRaphies
DOCUMents
PDF
HEADStones
VIDEOs
RECORDINGS (this one is the exception and TNG requires all the letters)
-----
The most common would be PHOTOS, DOCUMENTS, HEADSTONES and VIDEO, I've never used the others (yet).
I like the uppercase idea and it seems to fit with TNG. I believe you can create custom collections by specifying your own types but I haven't played with that.
Would you keep the uppercase keyword in the media text with the other keywords or remove them on export?

Ron

[rfield]

Just doing some test exports.

ALL~LMO - I get about 320 images out of 3,200.
ALL~ABS - I get to 1,200 and it stops at 78%. This is my fault, I get an "out of memory error". I checked the Object it stopped on and it's an mp4 video file. There is one individual, who was an actor and I have copies of three of his movies attached to him and they're about 300 MB each. I shouldn't really have them attached.

>Are OBJEct records @O4@ and @O5@ the only ones without CHAN, DATE, TIME tags at the end?
>If so, then that may be the cause and should be fixable.

I've compared between the project file and exported file and with or without CHAN, DATE, TIME doesn't seem to make a difference. I have found both types missing in the exported file.
I'm not sure why it switches from LMO to ABS and then back again in the same exported file. I'll keep looking.

[rfield]

A quick note.

I did get All~ABS to work and it exported all images without the large videos.
I can confirm that the primary photo setting works fine _PRIM Y

I'm testing _TYPE and I'm seeing issues.
Previously I tested with the images in separate folders.

There was a bug in TNG 10.1 with _TYPE that Darrin from TNG and I tracked down so I need to check my file versions on my test server.

I will continue to play with the All~LMO export.

Ron

[tatewise]

I have run some Plugin tests on Media OBJEct records without CHAN, DATE, TIME tags, and have replicated your symptoms, but it may only arise if the Media OBJEct record also has no NOTE, _ASID tags.

It is unusual for Media OBJEct records to have no CHAN, DATE, TIME tags, but your recent import from TNG probably has them missing and unfortunately FH does not automatically add them.

This will be fixed in the next Plugin version.

When I have some prototype updates for the other enhancements I will attach the Plugin for you to trial.

[rfield]

Sounds great Mike!

A little info on the gedcom history.

FH wouldn't import from RM7 or Legacy 8 without creating duplicate media links. I ended up with over 7,000 media items instead of 3,200.
I found that FH preferred absolute media links like FTM so I imported the file into Webtrees which I knew would export the Rootsmagic 7 file with absolute links.

I had also tried a gedcom from FTM at first but FTM 2014 has a bug in that it doesn't export media notes to gedcom. A bit frustrating

Ron

[tatewise]

Ron, the Plugin Store version was updated to 2.0 on Tuesday.
So I advise you update to that version before doing anything else.
It does not affect your mode of use with TNG, but helps with future updates.

Then try the attached Plugin version 2.0.1 dated 03 Jul 2015 with the following updates:

• Fixes OBJEct Media records without CHANge tags
• Supports _PRIM Y tag for first Individual Media
• Supports _TYPE tag for first all upper-case Keyword

You will need to use the Extra Options tab Reset these TNG Options button to get _TYPE tags.

You can always revert to V2.0 from the Plugin Store if necessary.

[ EDIT: Attachment deleted as now published in the FH Plugin Store as V2.2 ]

[rfield]

Hi Mike,

Wow, that's great! I just installed 2.0 first and will start testing 2.0.1.

I still have to figure out how to handle the _TYPE attribute.

TNG has two ways to look at media when imported.
If you use sub directories eg. Photos, Headstones, Documents, Media, you can supply the path from the gedcom file for the folders and on import TNG will know where to assign them. This requires you manage your media in folders and to copy the files in to the corresponding TNG folders.

On an import which uses a single media folder, if you put the files into the corresponding folders then the _TYPE is used since the gedcom has the same path for all media. Sorting the media in the single folder is done by adding a prefix to the filenames eg. p_ = photos, m_ = headstones, v_ = video and a_ = audio or recordings. Everything else is considered to be in the Documents folder. This way you can drop the files into the correct folders and _TYPE tells TNG which collection they should belong to.

I've also noticed that if you stick all the media into the TNG Photos folder and then within TNG assign media to collections that TNG copies the files to the correct folder. Unfortunately it doesn't do it on import.

I'll have to see what works best. Time to play.

Thanks Mike!

[tatewise]

Ron, you are not the first 'Exporter' to ask for the Media files to retain the same folder structure as in the FH Project. It might be feasible, but I initially took the simplistic approach of using one folder for the following reasons:

• I assumed the target product was secondary to the FH master Project, so the loss of certain 'details' such as Media folders/groups was unimportant.
• I am not sure that all target products support Media folders, so best avoided, and for reasons explained in the FAQ, all export modes should use the same folder hierarchy.
• If say TNG requires specific folder names, then FH would have to use the same folder names, and so would all other target products that you wanted to use.
• I would hate to have to implement a mapping dialogue for each target product to translate FH Project Media folder names into say TNG Media folder names!

[rfield]

Hi Mike,

I'm going to do some more reading on media usage with TNG. I agree that you shouldn't have to change the folder structure or map the output of FH according to any target application.

My first thought at the moment would be for me to use IPTC/XMP for tags in Windows. This way I can search by Tag and copy the correct files to each folder yet maintain the existing folder structure in FH.

Apparently single folders can be used but I believe there's a loss of the separation between types (photo, Document etc.)

Great news! The '_PRIM Y' tag works perfectly!! Thank you very much!

I'm just testing the LMO export and then on to _TYPE testing.

[tatewise]

IPTC/XMP is not supported by all image file formats (although most popular ones are OK) and is not supported by other media such as PDF, DOCX, etc...

If you change the Media folders on import to TNG won't that break all the Gedcom OBJE FILE links imported to TNG?

[rfield]

ABS and LMO exports work great now, and all images are being exported.
Also the _PRIM Y tag is working for both types of exports.

Thanks Mike!

[rfield]

If you use _TYPE then TNG looks in the folder that matches the type.
If you don't then TNG can use the path to the file to specify where it is from the following.
In the import settings dialog you can set the path to be removed (matched to) from the imported path.

Photos folder: c:\somefolder\Ron\photos\
Documents folder: C:\somefolder\Ron\Documents\

TNG will strip the path and look for the file in the folder.

When using the _TYPE it matches the file type to the folder and expects to find it there. I want to test some more as I've noticed that .pdf .mp4 and .mp3 are sometimes identified depending on the method used above even when they're in the photos folder. I'd like to look into this one more.

IPTC/XMP was a first thought when staring at the screen Most formats do support xmp but you're correct not all.

Just heading out to do some errands.

[tatewise]

Remember that whatever technique you use the Media files under discussion are the ones in the Plugin Export folder, NOT the FH Project Media folder structure.

This is because all the FH Part/Detail Frames have been used to synthesise cropped image files, as well as the original full image files, all with prefixes on the original filename.
Those prefixes are necessary not only to differentiate cropped from full image files, but also image files that happen to have the same filename in different Project Media folders.

[tatewise]

I've been digging around the TNG Media Import info on the Internet and found http://tng.lythgoes.net/wiki/index.php? ... a_-_Import and this is my interpretation of it, although I do not use TNG.

It mentions One Media Folder which is the norm for FTM (and others) and the FH Plugin Export folder.
So TNG Admin > Setup > Import Settings > Local foldername Path(s) must specify the same folder name for all media types.
I also read somewhere that you can simply set use filenames only.

This means that the media file folder path name cannot be used to assign files to Media Collections, but you don't have to, because we are proposing to use _TYPE tag values.

So I assume the _TYPE tag value can establish your Media Collections despite all the files being in one folder.
(In the same way IPTC/XMP tags can form subset collections.)
For this to work, every FH Media record must have an uppercase Keyword that matches a TNG Media Collection Name.
Otherwise, the media file will import and link OK, but will not be assigned to a Media Collection.

[rfield]

If you specify a path under import settings then TNG appears to ignore the _TYPE tag and expects everything to be in the single folder. This works but everything is in the "photo" collection. If you do designate a collection after your import TNG will move files from the single folder to the collection folder for you.
This would mean you would have to search by keyword, or some other method, select the files and convert to a collection and TNG would then move the files to the collection folder.

An alternate I was playing with today is to use a script. Since all the information we need from the export is in the gedcom file (using the uppercase keywords and _TYPE) I wrote a Powershell script that finds all the files, their _TYPE and then moves them into sub-directories within the export directory according to type. Then I copy the files to the corresponding folder and import the gedcom leaving the paths under import settings blank in TNG. This way TNG will read the _TYPE tag and expect to find the files in the collection folder.

In FH I'm noticing that the keyword drop down in Media doesn't always reflect the Case of the keyword although the plugin does. I was also unable to find a way to remove a keyword. I tried by unassigning it but the keyword remained even after restarting FH.

The only other issue I've run into is I've received a few "Not enough Memory" errors while trying to export a gedcom once I had added the uppercase Keywords to the 3,200 media items.
I restarted FH and they seem to have gone away for now.

So far the process is working great. I export, run the script, then move the files and import the gedcom.

I can provide the script if interested.

[tatewise]

Media Collection Import

For each Local ****Path(s) setting, have you tried leave this path field blank and check "Import file name only" for the last option on the page, as described in http://wilcockfamilyhistory.50webs.com/ ... _help.html.

Also http://tng.community/index.php?/forums/ ... th-gedcom/ says go to Admin > Setup > Configuration > General Settings > Paths and Folders and set the directory name to be the same for all of the media collections.

Then the implication is that the _TYPE tag value establishes the Media Collection of each Media record.
This is supported by http://tng.lythgoes.net/wiki/index.php/ ... a_into_TNG under the 5. Media Import with GEDCOM section.
Also http://tng.lythgoes.net/wiki/index.php/ ... ollections under Folder Name says "You may use an existing folder name, especially in sites where (typically because of PC genealogy application constraints), all of your media files are in the same folder."

So this implies that all Media Collections can reside in one folder and only differentiated by _TYPE names in the Media record.

Perhaps an Email to Darrin Lythgoe will resolve your problems.

Keywords

Can you post some screenshots of what you mean about wrong case?
To remove an individual Keyword from a Media record you just edit the Keywords and untick its entry.
The Keyword Update Plugin can be used to replace Keywords with any combination of Keywords including none.

[rfield]

So if you leave the paths blank and select the "import File Name Only" which is how I leave it by default the TNG will determine the collection type according to _TYPE or FORM eg pdf, mp4, mp3 and assign the media to a collection. Selecting a file, say a video will display a file not found error since the file is not in the media folder.

The second option of using the same path for all the folders indicates that all media is in the default folder "Photos". _TYPE is not being honoured in this case and everything becomes a part of the PHOTO collection. If you wish to have headstones, or media collection you would have to manually find and convert the files you wish to be part of the collection. TNG then moves the files to the target collection folder.

There's an FTM converter written for TNG. http://connergenealogy.com/LoadFTMintoTNG.html To use it, the author who had to deal with the same issue suggests using a prefix of p_ for photos, m_ for headstones and then sets the _TYPE according to the prefix. Only photos, headstones and media (video audio) have prefixes and the remaining are considered document. You then copy your files according to prefix and then import the gedcom.

I'll probably stick with the script which is working fine. Thank you for you help.

Keywords

I probably wasn't clear enough on removing a keyword. Say I misspell a keyword, "Dosuments", how would you get ride of it from the drop down even when it isn't assigned to any images.

I've included screenshots of the upper & lower case mismatch.

Document is lower case here.

Image1.jpg (30.42 KiB) Viewed 11693 times

Keyword Plugin.

DOCUMENT in the plugin

image2.jpg (33.22 KiB) Viewed 11693 times

[tatewise]

Media Collection Import

Forgive me if I am wrong, but I think you are missing something.
I suspect each Media Collection definition still uses a different folder as per the TNG defaults.
With those settings, then the folder path takes precedence when determining Media Collection.
As suggested in my previous posting, try setting every Media Collection to use the same folder, such as the TNG Media folder, then I think the _TYPE code will determine the Media Collection and you won't need a script.

I believe Media Folders and Media Collections are independent concepts, but the default setup of a separate Folder for each Collection gives the impression they are bound together. If consistently setup with only one Folder defined for all Collections, then _TYPE comes into play.

The FTM converter is probably necessary because the luxury of an Export Plugin to set _TYPE tags is not available, and it is neater to stick with the TNG folder structure for Media Collections.

Keywords

When I unticked Dosuments so it was not ticked in any Media record, then it automatically vanished from the drop-list OK.

As far as FH is concerned Keywords are not case sensitive, and Document is one of the five predefined default FH Keywords that will not vanish even if not ticked in any Media record, and so will always be displayed in Title format. Use DOCUMENTS instead, which is the recommended TNG Media Colection name.

[rfield]

So I tried setting all the import paths the same and all the folder paths the same. Moved all the media into the \photos folder, the one I set the same for all the collections. TNG didn't recognize the _TYPE tags and made them all photos.

Stick with me..

So I thought I have an old version 9.2 setup let's try that.

I set all the paths the same and set the folders to all point to photos. Moved all the media to \photos. I imported the gedcom and it worked! Collections set with the _TYPE and all in the same folder with working links.

So I went back to the latest version. blanked out all the import paths, left the folder all pointing to \photos and imported the gedcom. It worked!

Bet you're thinking I must be pretty thick by now since you told me to try that a couple posts ago! Apparently every time I read your post I kept thinking Local folder path under the import settings.

I give you a lot a credit for sticking with it when I was ready to pack it in and use a script. Thank you!

Keywords

It figures that the one I was trying to get rid of was the lower case Document so I could use DOCUMENT instead. I saw another keyword that wasn't being used disappear but couldn't figure out why I couldn't get rid Document. I will use DOCUMENTS instead.

One thing, I haven't started to look at yet is how FH handles marking living (or dead) individuals.
TNG understands "1 _LIVING N" and I use to convert "1 DEAT Y" before import.

I know there's a living flag but I haven't figured out where it fit's in to the export or how to set them all and I've also seen "1 DEAT Y" and "1 DEAT" in the export.

FYI, Source Method 1 images work properly in TNG as do place images. FTM and Rootsmagic didn't support place images in gedcoms!

[tatewise]

Media Collection Import

I'm glad that worked in the end, and plan to update plugins:help:export_gedcom_file:tng_the_next_generation|> Export Gedcom File ~ (TNG) The Next Generation with the _PRIM Y and _TYPE tag Export Details and the TNG settings Import Details.

Could you please post screen-shots of TNG settings screens for non-standard options such as clearing all the paths.

Keywords

This all seems to be resolved, and the advice above will mention the Keyword Update Plugin to use in conjunction with the _TYPE tag.

Living Individuals

The FH predefined Living and Private flags can be added to any Individual record to utilise standard privacy options in various features.
To set or clear a flag, select the Individual record(s) and use Edit > Record Flags.

Use View > Standard Queries > Estimated Alive At with current year to list every Individual alive now.
Then use the Query Menu > Set Flag on Row Records and choose the Living flag to set in bulk.

The Knowledge Base help page above explains how to get flags into TNG.

The 1 DEAT tag is the GEDCOM code for the Death Event and is followed by Y only if no Date, Place, etc, details are recorded. (Just like 1 BIRT is the Birth Event, and 1 MARR is the Marriage Event, etc.)

Local Media Objects

Sounds like my fix for Media records without CHAN, DATE, TIME tags is working OK.

Have the "Not enough Memory" errors stopped?

[rfield]

I have attached a couple screen shots of the two areas. Please let me know if you need any others.

I had used the Estimated Alive At but wasn't sure if I needed to add to a named list first or not. Thank you.

Now if I add the _FLGS type to TNG do you know if TNG understands that it means living? I'm going to try it anyway so I guess I'll find out.
I'll also try the _USED by adding it too.

I did notice that since I separated the address from the location that I'm no longer able to assign a cemetery in TNG to a specific cemetery location. I think I just have to move the cemetery address back to the place and I should be good. This lets me add media and gps too.

I have a lot of reading and playing to do to become familiar with FH. Some clean up to do too. Backing up often in case I blow up my data

I have not seen any more "Not Enough Memory" errors. If I add back large videos I can get it to happen but it's been fine.

Yes, the Media fix appears to be working fine for LMO exports and all media is being exported.