* Import gedcom with photos from online url

[tatewise]

OK, thank you, that explains everything.

The plugin currently only operates on Media Records and not Local Media Objects (LMO) as in your example.
My reasoning is that when a GEDCOM is imported as a Project the LMO are converted to Media Records by default.
I believe that to be the case in both FH V6 and FH V7.
How did you manage to prevent those LMO from being converted?

FH V7 is very aggressive about Validating Media and strongly encourages LMO to be converted to Media Records.

The message probably needs improving to mention that only Media Records are checked for URL.

[Valkrider]

Mike

I didn't do anything I just used the standard Gedcom import option to pull this gedcom into a new project where I could look at everything before combining the two projects. The gedcom snippet I provided was from the gedcom I was sent not from the FH one I will check that and attach it.

This is the FH gedcom

0 @I5@ INDI
1 NAME Emma Jane Ashbee //
1 SEX F
1 FAMC @F1@
1 FAMS @F3@
1 _SVG_P 5,2,False

So as you can see FH hasn't done a particularly good job of importing the gedcom as it hasn't even identified the surnames correctly. Don't forget I am still on FH v6 not v7. The original gedcom I was sent was a 5.5.1 not the 5.5 of FH v6.

[tatewise]

FH V6 does not take any notice of the GEDCOM version.
The supplied GEDCOM does have an 'interesting' style of Record Id.
The LMO is GEDCOM 5.5.1 with the 3 FORM tag subsidiary to the 2 FILE tag.
It is very unusual for FH to mishandle the NAME tag like that.

Your attached FH GEDCOM does not illustrate how the Media has been imported.

[Valkrider]

Mike the media has not been imported into FH that is the issue.

The gedcom spec says that the Record ID can be constructed of any characters up to 22 characters long. I have come across some really weird ones with people sending me examples for testing against my Wordpress plugin.

[tatewise]

You misunderstand. How did FH import the LMO and convert into a Media Record?

I've taken your snippet of GEDCOM, incorporated it into a tiny GEDCOM file, and imported it into both FH V6 and FH V7.
That worked as I expected. They both converted the LMO into a Media Record with the URL as the File link.
Then my plugin tried to download from that URL but naturally failed as it does not exist.
If I substitute a valid URL then it downloads the media image and links it to the Media Record.
I would expect my plugin to download your genuine URL successfully in the same way.

BTW: In both FH V6 and FH V7 they imported the 1 NAME Emma Jane /Ashbee/ perfectly correctly.
So I cannot explain why in your case both // moved to the right. Very odd.

This is how the Project GEDCOM looks and is essentially the same for both FH V6 and FH V7:
0 @I1@ INDI
1 NAME Emma Jane /Ashbee/
1 SEX F
1 FAMC @F2@
1 FAMS @F1@
1 OBJE @O1@
1 _SVG_P 5,2,False

FH V6 Media Record:
0 @O1@ OBJE
1 FORM jpg
1 _FILE https://xxxxxxxxx.com/Images/Ashbee_Emma_Jane_b1834.jpg

FH V7 Media Record:
0 @O1@ OBJE
1 FILE https://xxxxxxxxx.com/Images/Ashbee_Emma_Jane_b1834.jpg
2 FORM jpg
1 _KEYS Picture

If you used File > Gedcom File Tasks > Open Gedcom File... to load the supplied GEDCOM then the LMO are not converted to Media Records, but that is not a Project and my plugin should have said it only runs in the context of a Project instead of the message you posted. So I'm mystified how you opened the GEDCOM, did not convert LMO to Media Records, yet it was a Project.

[Valkrider]

Mike

Given what you said I deleted the existing project and created a new one.

This time the project imported and displayed the surnames correctly between slashes.

Your plugin also runs and works fine

Screenshot 2021-03-14 at 18.57.27.png (39.32 KiB) Viewed 4512 times

Thanks this plugin is an excellent addition to the FH toolkit.

[tatewise]

Great!
A more intensive test is with Family Tree GEDCOM downloads from Ancestry and FindMyPast.
I get some success with my downloads but needs some independent testing with other users' trees.

[Valkrider]

I will try Ancestry and FMP in the next few days and report back.

[tatewise]

Attached is a slightly updated Download Online Media plugin Version 0.7.1 Date 15 Mar 2021.

It has the WARNING suggested by Nick Walker, recognises MyHeritage download URL, checks the Ancestry URL template supplied by users, and adds a few minor refinements to make it more robust.

[Beerbadger]

Hello Mike,
I am a new user of Family Historian v7 and this forum.
I tried FH for the 30 day trial and was so impressed that I bought it. I particularly like the user developed plugin capability.
Anyway I saw your posts about the download of media from Ancestry and was immediately interested.
I have downloaded v2 of the plugin this morning and tried it, as I have lots of Ancestry URLs in my Media, presumably because I created my FH Project from an Ancestry GEDCOM download.
I dont know where my Media Gallery is in Ancestry so I chose a person opened her gallery, picked an image and opened it full screen and used the URL from the browser address bar.
The plugin did not work using this URL, giving a 'Media Link does not match required URL format' error.
I then right clicked on the image and used the 'Copy Image Location'. This caused the plugin to operate, but it failed on all 93 images.
I am using Firefox v86.0.1.
I thought that I would let you know by way of feedback in case it helps with development.
Cheers
Daryl

[tatewise]

Welcome to the FHUG Daryl and thank you for the feedback.
It sounds like the Plugin Help and Advice needs to be more pedantic and the URL format check more rigorous.
The danger of being too rigorous is that it excludes valid URL, i.e. false negatives.
Unfortunately, Ancestry is more awkward than the others and needs that URL template to access the media.
The others can use the URL in the Media records as they stand.

When providing this kind of feedback please include specific examples.
e.g. Provide a sample URL from your downloaded Media records File box, and the URL you entered into the Plugin.

I suspect you just fell short of the required URL. You were almost there.
With the image displayed as you described it would have had a toolbar at the top and a side-panel of details.
Instead of right-click Copy Image Location if you had used View Image as requested by the Plugin and used the resulting URL in which only the image is displayed without any toolbar or side-bar, then I think it should work.

BTW: With the Ancestry Family Tree displayed, click the top left droplist where the tree name is shown.
That offers Tree Overview, Media Gallery, All Hints, etc, and that Media Gallery option displays all media.
However, your route via an individual Profile gets there too but needs more clicks.

[Beerbadger]

Hi Mike,
Thanks for your welcome and speedy response and the hint of where to find my Ancestry Media Gallery.
I went to media gallery right clicked random image and chose 'View Image'. This brought me to a page with no Ancestry side bar or top bar. I copied the URL from the browser address bar into the plugin. Unfortunately it still failed on all the images.

The URL I used in the plugin is https://mediasvc.ancestry.com/v2/image/ ... axSide=160

An example of a URL I have in FH View All Media is this
Media\URL\https mediasvc ancestry com v2 image namespaces 1093 media 675fd69f-172c-4b1f-8481-1eb26a48fa70%3Fclient=TreesUI.url
Which I believe must have been entered by the plugin because on the first run, I think, it said that it had succeeded with 5 and 93 failed. This image is just a Firefox logo in the FH View All Media.

I am not sure where you mean by 'your downloaded Media records File box'
Cheers
Daryl

[trevorrix]

I am beginning to experiment but am falling at an early hurdle. My GEDCOM downloaded from an Ancestry family tree contains one media record as a test.

1 OBJE
2 FILE http://trees.ancestry.com/rd?f=image&gu ... 495&pid=96
2 FORM jpg

But FH7 does not import that record into its project GEDCOM. Please advise.

[tatewise]

@Daryl: The URL you entered into the Plugin this time is correct.
However, I suspect the previous runs of the Plugin may have upset the Media URL in the Project.
To see the File box, double-click one of the Media Records listed on the left of the Media Window.
That should open the Property Box (like an Individual Property Box) with boxes labelled Title, Date, Format, File, Note, etc.
The File box should display a URL similar to:
http: //trees.ancestry.com/rd?f=image&guid=64940e25-d446-4dba-919e-a19d7742d585&tid=15707378&pid=1
Click on any other Media Record to see its contents in the Property Box.

If not then you need to revert to an earlier Project GEDCOM (or import the GEDCOM into a Project again).
Use File > Backup/Restore > Revert to Snapshot... and choose a Snapshot Taken earlier to revert the Project GEDCOM to that snapshot.

BTW: After running a Plugin, to undo its changes use Edit > Undo Plugin Updates before closing FH.

@Trevor: The FILE URL is good, but how are you importing that GEDCOM into FH?
Your snippet shows a Local Media Object (LMO) and not a Media Record, and Colin had similar issues yesterday.
It is important that you use File > Project Window > New Project as is the usual advice for importing an Ancestry GEDCOM into FH. That should convert the LMO into a Media Record that the Plugin can use.
Do NOT use any other methods such as File > Gedcom File Tasks, File > Merge/Compare File, or editing the Project GEDCOM file. Those methods will NOT convert LMO to Media Records.

[Beerbadger]

Hi Mike,
thanks again for your continuing assistance.
I now know what the 'File box' is.
Almost all the URLs in my File box for media are like the example you gave, the one I quoted is an odd one out. However, I can fix it when I have this plugin issue resolved.
When I double click the media name on the left and get a floating file box, all the files say 'File not found' in the place where locally stored media shows the thumbnail of the image.
If I have the Media record show on the right (fixed not floatin) rather than the thumbnails, most of it is blank, but it shows the image title and the URL. Alongside the URL it has a hyperlink 'web location'. If I click that I get to the Ancestry page with the image.
Just to check my understanding of what the URL does - I thought it should download the image from Ancestry, is that correct ? FH clearly knows where the image is, but the plugin is saying fail on every image.

The plugin is still saying failed for all the media it tries to load when I run it. I left it to complete and got the attached response (5 files downloaded,93 failed)
Have you any other suggestions for me, please. I am desperate to synchronise my Ancestry media with my FH so thatr I can then continue with FH as my main documentation.

Another observation that seems odd is that when I press the elipsis at the end of the box with the URL in it in the File box it takes me to my backup location with the last few characters of the URL in the filename field in a box headed 'Select file'.
Cheers
Daryl

[tatewise]

Thank you for sticking with this.

From your description, it sounds like you have ticked the URL Shortcuts? option in the opening dialogue where you enter the Media Link URL. I suspect that is what you are seeing for 93 failed downloads. The Plugin has created that URL hyperlink in a format that FH recognises. If clicking that displays the image then we must be close to achieving a successful download. It is a matter of discovering the exact URL that Ancestry likes.

At the end of the Plugin what Failure Mode does it give in the Result Set for the 93 that failed?

Is there anything that differentiates the 5 successes from the 93 failures?
e.g. You uploaded the 5 media but the 93 are from Ancestry records? The 93 have an unusual file type?

Would you mind posting a few URL from the File box of some that failed prior to running the Plugin?
Could you also post the URL of the Ancestry pages that the URL hyperlinks display after running the Plugin?

Regarding your last point, that is FH providing a dialogue for you to manually select the file that FH cannot locate.
FH does not recognise it as a URL and treats it as a malformed Windows file path.

[Beerbadger]

Morning Mike,
On the contrary, thanks to you for your continuined help.
This morning I ran the plugin again and got 7 successses and 98 failures.
The mode of failure of all 98 is 'http status Not Found 404'.
A couple of examples :
http://trees.ancestry.com/rd?f=image&gu ... 67&pid=520
http://trees.ancestry.com/rd?f=image&gu ... 667&pid=97

The seven successes this morning appear to be all local media. I sorted the All Media view in FH using date updated and find today's date against 7 items with the 'Media\....' file name.
I did run it this time without being logged into Ancestry and using the 'remembered' seed URL from my runs yesterday.
Cheers
Daryl

[tatewise]

Yesterday, when you clicked the URL hyperlink and successfully displayed the image, presumably that was in your default browser and you were signed in to Ancestry.

I tried your example URL with your Media Link URL and the resulting URL Shortcuts displayed the same dummy image and not the required image.

I wonder if successful downloads do not need you to be signed in to Ancestry but the failures are because they do need you signed in. The 'snag' is that FH internally uses the Internet Explorer Shell as required by Microsoft. So even if you are signed into your default browser, as far as FH & IE are concerned you are not signed in.

So please run Internet Explorer and sign in to your Ancestry account, and then run the Plugin on the original URL.
Fingers crossed.

[Beerbadger]

Hi Mike,
I dont trust Internet Explorer, I have never used it on this PC previously.
Ancestry gave me a message saying that some functionality would not work with IE and that I should update my browser.
However, having said that I did the experiment and got 98 failed and no successes this time.

Cheers
Daryl

[Beerbadger]

Hi Mike,
I have had a small success.
Using Firefox, I created a new Ancestry Tree with just one person in it and put one random photo in it. I then followed your process for seeding the Download plugin, while still logged in to Ancestry, and it worked
So I will now copy a few of my real tree people into this test tree and do the download again to see what happens.

Cheers
Daryl

[tatewise]

Ok, let me think about it and try some of my own experiments. The difficulty is that I don't usually subscribe to Ancestry so will need to use a free trial account.

Going back to some of my earlier questions, are the successful downloads your personal images, whereas the failures are images only available when signed into your subscription account? Is that correct?

[Beerbadger]

ok, next chapter.
I added one more person into my test tree from my main Ancestry tree, tried to run the download and got the 'Download website not found in any Media Records'.
Now it keeps giving that error message.
Just looked at the media record and it doesn't contain a URL, it is just stored locally in the FH 'Media\...' location.
Daryl

[tatewise]

Darryl, it helps if you explain exactly what commands you used so I don't have to guess
I presume you used File > Merge Compare/File... to incorporate the extra person.
That does not convert the Local Media Objects (LMO) used in the Ancestry GEDCOM to Media Records preferred by FH.
So use the File > Validate... command and FH will offer to convert them.

[Beerbadger]

Hello Mike,
Yes, the successful downloads are my personal images, but they were stored locally anyway. I don't even know why they should have been updated. I am not sure they are even on the Ancestry site.
Is that intended ? After using the download plugin should the images be disconnected form their Ancestry origin ie no URL just a local address on my disk ?
Cheers Daryl

[tatewise]

Those images must exist online in your Ancestry Family Tree otherwise they would not appear in the GEDCOM as URL.
You must have uploaded them sometime in the past, or maybe you sync FTM with Ancestry.
Go check the Ancestry Media Gallery and they will all appear there online.

After running the Plugin those downloaded Media Image files are inside the Project folder structure in the Media folder.
The Media record File box will say Media\<filename>.jpg which is the Windows local path to the file in place of the URL.

Your original personal images stored elsewhere on your PC are not involved in the process.
The Plugin knows nothing about them.