* Export Gedcom File new features

[tatewise]

There have been several recent discussions about this Plugin, and the attached prototype hopes to fix most of them.
If all goes well it can be published in the Plugin Store soon.

ATTACHMENT Export Gedcom File Version 3.2.4 Date 06 Jun 2017 in ZIP file. [Now Deleted]

This version :-

1. Fixes the initial Window freezing issue reported by Ron
2. Fixes the Ancestry Other Sources issue reported by David
3. Adds a bulk Media ZIP file upload for FindMyPast (see the Help & Advice for FMP Output Format)
4. Includes all the other new features added recently

Please let me know of any problems.

[David Potter]

Hi Mike

Thanks very much - I'll give it a test.

[ronk]

For my usage, Export Gedcom 3.2.4 is ready to become 3.3! Great job. All worked as it should, both LFT and TNG, including passing along the PLAC geocoding.

And the freeze; amazing you found that bug. And yes, the 3.2.4 version fixed the errant .dat file as well.

[tatewise]

Would anyone care to try the FindMyPast bulk Media upload?
You do not need a subscription account, as a free account in FindMyPast (and Ancestry) supports uploaded trees.
Check the plugins:help:export_gedcom_file:fmp_findmypast_family_tree|> Export Gedcom File ~ (FMP) FindMyPast Family Tree for details.

[jbtapscott]

Mike - I'm happy to have a look at the FMP Media upload but is there a config file of my current settings that I can copy / save before switching to the FMP export options?. I have "tweaked" a number of the Gedsite Export default settings and would not wish to loose them. Thanks

[tatewise]

Brent, thank you, you (GST) GedSite settings should be safe, but create a File > Backup/Restore > Medium/Full Backup just in case.
Then Choose the GEDCOM export mode: as (FMP) FindMyPast Family Tree.
You will need to change the Multimedia conversion to anything other than (WIPE~ALL) as explained in the Help & Advice identified below, and you will have to set some Media Keywords to FindMyPast to select those to upload. This is because there are copyright issues with Media on public website trees, so the Plugin cannot simply upload everything.

[jbtapscott]

Not had much success I'm afraid Mike!.

I used the Keywords plugin to update my Media records, and, after installing the 3.2.4 Plugin, I used the default settings for FMP and ran the Export. When finished, I followed the instruction in the Help to create the compressed Zip file (approximately 280 media files). I then logged in to FMP and ran the Gedcom import process. At its conclusion, I again followed the Help instruction to import the Media zip file.

The first couple of times I ran this I received (eventually) a 404 File or Directory Not Found error. As my internet upload speed here in Greece is very slow, I thought this might actually be a timeout issue, so went back to FH and removed the Keywords from all but six Media records. I reran the Export and FMP imports again. This time, the Media import completed and I got the message about the records being processed / might take a few minutes, etc. I waited ten minutes and then refreshed the Media page in FMP. There were six placeholders displayed, with Titles matching the six records I inserted the Keyword into, but no "pictures".

I then tried the "single" import of one of the media files and this got matched correctly with the relevant placeholder.

I've tried this several times now and always without success. I also tried changing the "Change Multimedia Conversion" option from FILE~ABS to FILE~REL on the off-chance that might work but again no luck.

I'll give it 24 hours in case there is an issue at FMP and try again.

As an aside (and not relevant to the above), I noted that the "Empty Export Folder" option did not delete the FindMyPastZip folder when I used it - I manually deleted it each time I ran the export.

[jbtapscott]

Ignore the above Mike!. It's taken well over an hour but the media images are starting to appear within FMP.

[tatewise]

I am glad you are now getting some success Brent. There seems to be a knack to it, and you must be patient.
280 Media was a bit ambitious for a first go!

I've only ever tried it with a dozen or two Media images, and the slowest step was uploading the ZIP file, then it did take a few minutes to associate the images with the grey place-holders.

Beware that the correctly associated images have a full file-name including file-type as their Title.
If you get images with the file-name but without the file-type, then they have NOT associated correctly, and need a workaround that I can explain.

I have never got single Media uploads to associated with the place-holders. They appear without the file-type as above.

I probably need to add some advice about the time it all takes, and to be patient.
It might help to create a number of separate ZIP files and perform the upload in batches?

You are right about the Empty Export Folder option. I will correct that in due course.

It is important to clear the folder, especially when changing Multimedia mode and changing Keywords, otherwise the mix of files will be wrong.

If you try one of the (****~ABS) modes, then your Preferred photo per Individual should automatically become their Profile image in FMP.

BTW: I presume you know how to open the Profile of an Individual in FMP and see the attached Media images?

[jbtapscott]

Thanks Mike - points noted.

Yes, it was a tad ambitious loading that number but I felt that if I was loading my tree, that is the sort of numbers I would be looking at!. Yes, I have been using the Profile of the Individual to look at Fact / Media connections and viewing the images.

I'm a little unsure about your comments in respect of the Media record title. After loading the Gedcom , and before importing the media files, the Media page on FMP shows six grey placeholders but the Title for each of these is the full file name and extension/file type (viz .jpg). If I manually import a single file (which, as you say, does not get associated with the placeholder), then the title does not include the file extension - indeed, there appears to be a truncation of the Title name as well - my file called "Tapscott, Marjorie H - Birth Certificate (Photocopy).jpg" created a Media record with a Title of "Tapscott, Marjorie H - Birth Certificate (Photocop" (not an issue with the Export Plugin but something which I would take up with FMP if I did decided to load my tree on a more permanent basis). The media record (with the truncated Title) was correctly linked to the Individual / Fact.

I'm still finding that it takes over an hour for the six images to appear on the Media page, despite frequent refreshes.

I will carry on testing tomorrow.

[tatewise]

Brent, how are you uploading the single Media file?
Via the Individual Profile > View All Media > + Add Media route?
Or the Tree Settings > Media > + Add Media route?
The former will certainly attach to the Individual, but the latter I believe will not.

I am surprised at the time it is taking for only six images to associate.

BTW: The filename truncation is not something I have seen, but my filenames are not that long.

[jbtapscott]

I have tried both ways Mike - as you say, the import via Tree Settings - Media - +Media route did not auto attach to the Individual Fact.

I do find it a bit disconcerting (worrying?) though that the zip file import can take so long after it is completed for the contents to appear on the Media page - indeed, more than half of the imports I have run did not even get on to the page. As a minimum, I would have expected some sort of FMP error message / status log to show why an entire import goes "missing" when the first indications are that the import has been successfully completed. Again, not an Export Plugin issue , but something I will take up with FMP.

[tatewise]

The FMP single Media upload is a long-standing feature, and similar to other websites such as Ancestry and ZoomPast.
But that is a long-winded process, and I was hoping the bulk Media ZIP upload would be a more convenient alternative.
However, it appears to be somewhat unreliable, and there is little or no advice on how best to use it.

Although, quite often the uploaded images do not all match the place-holders, and create independent images, I have not experienced the total going 'missing' images that you have.

A couple of years ago the thread convert linked faces (12058) had this advice from Philip Moir - Technical Lead Findmypast Family Tree Team:

Normal GEDCOM import does not generally include image files but does references so it creates image slots with all the referenced data. You can then use the ZIP upload from the Media page under settings to load your images. This upload will try to match the images against the "blank" image slots the GEDCOM created and re linking to all the places you would expect in your tree. If the GEDCOM import had facial coordinates for a person within an image, the ZIP upload will also detect these and correctly pick out the profiles for the people.

I can find no details about the GEDCOM format needed to best support the bulk ZIP upload, nor the format for "facial coordinates for a person within an image.

I experimented with GEDCOM exports from FMP, and had to adapt the Plugin to use custom FMP GEDCOM tags & values to get the bulk upload to work at all, as standard GEDCOM 5.5 Media links did not work at all well. Any facial coordinates entered into FMP do not appear in the exported GEDCOM, so there are no clues as to how FMP detects those in an upload.

I have E-mailed Philip Moir with no reply yet.

[jbtapscott]

Thanks Mike. I have emailed FMP support but all they have come back with is instruction about import of single media files - I have replied repeating that it is the bulk zip import that is "loosing" my files.

[tatewise]

Yes, my experience with those support responses is similar.
They seem to give a knee jerk response to what they think is most likely to be the most popular scenario.
You have to repeat and emphasize the exact details of your problem.
I recently had a similar dialogue with Ancestry about GEDCOM Download and they kept advising about GEDCOM Upload ! ! ! !

[jbtapscott]

I have had a further reply from FMP support

"..... You would not be able to upload a bulk media zip file. As we do not support this when uploading to the Family Tree. You would be required to up load the images individually to your tree. ...."

I have replied with a snippet showing the menu options for the zip file import. I await the next response with interest!

[tatewise]

You could also supply the quote from Philip Moir that I posted yesterday saying ZIP upload is available.

[tatewise]

Brent, I have also reported the bulk ZIP Media upload problems to FMP and awaiting first response.

Regarding the long Media name truncation you mentioned last Thursday, I have conducted an experiment.
The truncation only applies to the caption shown underneath the image in the Tree settings > Media page.
If you use the Edit option, then the full name is in the Title box, but must scroll right to see it all.
In the Individual Profile page, when you hover on the attached image, the Title is shown in full.

The attached Export Gedcom File Version 3.2.5 Date 11 Jun 2017 [Now Deleted] fixes the folder problems you mentioned.
BTW: I have changed the FMP Zip sub-folder to ZipFMP to match the recommended ZipFMP.zip filename.
The Empty Export Folder button now removes the ZipFMP sub-folder too.
If the FMP Multimedia conversion mode has been changed when the Export GEDCOM is performed, then the ZipFMP sub-folder is emptied so it only ever contains the Media associated with the chosen mode.

[jbtapscott]

I have had a couple of replies from FMP support - the first asked me to send them the Media zip file I was trying to load, which I did.

The second response was:

"It appears the 6 files have already been successfully added to your tree, these appear within the media section -

http://tree.findmypast.co.uk/#/trees/5b ... /all-media

They wont attach to the individuals when added in bulk just the main media page. ..."

All that link shows is the page on my family tree with the media placeholders (viz. what is created when you import the Gedcom file). I have responded with Philip Moirs comment from your earlier post.

[jbtapscott]

I feel I am going round in circles with FMP support!. Their latest response is:

".... I have contacted our technical team regarding the issue. They have confirmed that if the source references an accessible path, we will then try to match the uploaded images from the uploaded zip (as we state on the site).

However, the filename and format (.jpg, .png, .pdf, etc…) should match from the GEDCOM file with the uploaded source from the zip file. Then we try to update the existing reference with the new source reference.
..."

As far as I can tell, the plugin does exactly that, so the file name in the Gedcom does match the file name in the Media zip file. I have made this point to them and await their response.

[tatewise]

I am in a similar protracted dialogue with FMP support.

First they discarded the Philip Moir statement saying he does not work for them any more, and that matching Zip Media was unlikely to work.

I replied saying the fact that Philip Moir has not worked for Findmypast for some time is irrelevant. He was a bona fide representative of Findmypast who described their ZIP Media upload feature. I asked if they were saying that as “Technical Lead Findmypast Family Tree Team” he did not know what he was talking about?

I also quoted from their own web page "If your tree originated from a GEDCOM import, we will try to match the uploaded images with the original media items."

That prompted them to talk to a tree specialist colleague saying "I'm just Customer Support rather than an engineer or tree specialist."

The specialist reply focussed on the word try and that "there is no guarantee the source matching will be successful." but did say "The filename and format (.jpg, .png, .pdf, etc…) should match from the GEDCOM file with the uploaded source from the zip file."

I have replied again saying that the format of the GEDCOM file can significantly affect the chances of successful matches.
I gave them an example of Standard Gedcom 5.5 that does NOT match well, and a Findmypast style GEDCOM using their custom tags (as used by the Plugin) that does match very well for me, but less well for you.
I have asked if any other 'magic' formats will help with matching, and why standard Gedcom does not work well, despite meeting the filename and format criteria stated above.
Also why are those 'magic' formats not published, since they appear to be important.
I also referred back to the Philip Moir statement "If the GEDCOM import had facial coordinates for a person within an image, the ZIP upload will also detect these and correctly pick out the profiles for the people.” and asked about the Gedcom format for those facial coordinates.

I await with bated breath!

[tatewise]

The support dialogue is now focussing on why the FindMyPast style of Gedcom works so much better than Standard Gedcom, given that all it should be doing is matching file-names, and that users would expect to use Standard Gedcom without the special FindMyPast style that requires :-

• 1 TITL and 1 FILE values must be identical and include file type
• 1 FORM JPG must use upper-case JPG regardless of file type format
• 1 _ASTTYP 1 and 1 _ASTPERM 4 asset tags must be present

Also 1 _ASTDESC must be used in place of 1 NOTE to get descriptive notes accepted but no impact on file-name matching.

The support team member has been passing messages to and from a tree specialist named Attila, but from now on we will be in direct communication.

[jbtapscott]

I have also just received an email from Attila, asking me to forward copy of the Gedcom and Media zip files (which, in fact, I have already sent to FMP Customer Support, but will do so again!).

In his email he states "...This feature was implemented by Phil Moir and its development was discontinued, but I will do my best to solve this issue for you and match all of your missing references...."

[tatewise]

I seem to have lost contact with them as I have not heard from Attila despite a follow up message saying I am still waiting.

You need to make it clear to Attila that we are not looking for a single fix just for your one upload, but for a general fix that will work for all users and all uploads repeatedly.

[tatewise]

Brent, could you please checkout the new version in Export Gedcom File with Full v Brief option (15110).