* Geocoding stopped - almost all the time

Questions regarding use of any Version of Family Historian. Please ensure you have set your Version of Family Historian in your Profile. If your question fits in one of these subject-specific sub-forums, please ask it there.
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Geocoding stopped - almost all the time

Post by Sadie »

I have recently upgraded to Version 6.2.7, and was particularly looking forward to using the geocoding function. My database has about 39,000 people in about 13,000 families, so I was aware that this might be a slow process. My computer runs Windows 10.

My plan was to geocode all recorded locations, and I did see that the geocoding process ran successfully for about 10-20 minutes or so the first time I started it. Every subsequent time I started the mapping process, it ran for no more than 1 minute - often just a few seconds - before stopping, and displaying an error message. At that point, all the uncoded locations changed their icons from the clock symbol to the grey question mark.

The error message reads: "Geocoding service has been stopped. Server busy."

Estimating from a few sample "screenfuls" I estimate that about 10% of the places have been coded. I can find no pattern to suggest why some have been coded and some have not.

Have I set up something incorrectly?
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

I suspect the order they have been geocoded is the Record Id order of the Place records.
To confirm that, in the Records Window on the Places tab, click the Record Id column heading to sort into order, and review the Latitude/Longitude column.

It is possible that the geocoding servers were busy, or that they apply quotas so that one user cannot hog the service.
The quotas will probably be reset at regular time intervals such as every hour or every day.
I know that the Google Maps Geocoding service used by my Map Life Facts Plugin has such quotas.

Try again another day and see if it is any better.
Instead of using Map All Places, open the Places tab mentioned above, then select a subset of Place records and use the Map Locations of Selected Place Records option.

FYI: The FH Help > About Family Historian refers to Map Search and Geocoding which refers on to https://www.algolia.com/ and https://opencagedata.com/ geocoding services. The latter in its FAQ says "The free trial tier is limited to 2,500 requests per day." I assume FH usage is as a paying customer ~ but not certain.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

Mike - Many thanks for your contribution.

I can confirm that they locations do appear to be coding in order of record ID, so that part of it now makes sense. I can now also see that the software has made a pretty good fist of finding the right locations.

I'll keep a note of the record ID which has been reached today, and see if it creeps up gradually over the next week or two.

Your input is much appreciated!

Sadie
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

I belatedly added some advice about geocoding a subset via the Map Locations of Selected Place Records option.

I also suggest on Tools > Preferences > Map Window to tick Block Refresh of Non-tentative Geocodes.
Otherwise all the Places that have already been geocoded will get refreshed unnecessarily and waste server time/quotas.
Also if you manually adjust the Latitude/Longitude it will get reset by the refresh.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

Very good points! Have implemented them.

I can now also see that, on those occasions when the coding stopped almost immediately, about 100 more locations were coded. Looks as though I will get there eventually.
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Geocoding stopped - almost all the time

Post by Jane »

I suspect you have addresses in your place fields, so have very large numbers of Place records, so are working the GeoCoding service hard.

As Mike suspected the service is limited to prevent a single user from hogging the service. You have probably reached the limit for a single day.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

Jane - thanks for that comment. I should perhaps have said that the great majority of my places are listed at parish level, or equivalents overseas. Only in my early stages of family history did I record addresses in the Place field, and I have weeded those instances out over the years. My addresses are now stored only in the address field.
Consequently, for my ~39,000 entries (I have been researching for 26 years) I have ~5,000 places recorded. Not having compared notes with anyone, I was not sure if 5,000 places was large or small!
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Geocoding stopped - almost all the time

Post by Jane »

That is a fair number, so you will just need to be a bit patient with the built in geo-coding.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

Yes, that is quit a lot of Place records, but not considering your number of Individual records.

Jane, do you know if the FH geocoding service is a subscription service paid by Calico Pie, or does it rely on a free option, or is it some other arrangement?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

I would be interested to see the answer to this. Although I am keen to use the geocoding, I don't want to use it unfairly or disproportionately.
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

Just go ahead an use it ~ that is what it is for.
Your heavy use will only be for a short while, then like everyone else it will only be when you add new place names.
As you have found, there are limits in place to prevent excessive usage.

Just do a batch of say 1,000 a day.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Geocoding stopped - almost all the time

Post by Jane »

I don't know which service for sure, but as none of the services even paid have unlimited access, I can see the logic of locking users to a limit per day, most people would not notice as it's only where there are a very large number of places to geocode that the limit seems to kick in.

The fact in 4 years Sadie is the first person to notice the problem suggests the limit is fairly generous.

If someone was in a hurry they could always use your Maps Plugin rather than the standard Geo-code option.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

No they can't, as I said, the Map Life Facts Plugin uses Google Maps Geocoder that also has quota limits (about 2,500 per day I think), and always has, unless you are personally prepared to pay. Although that is in addition to whatever the FH geocoder provides.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

For reference purposes, I thought I'd let you know that when I switched on my computer today, and let it continue geocoding my full list, it added 99 more records' latitude and longitude before stopping. I will see over the next few days if that is typical.
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

Are you selecting Place records without Latitude/Longitude and using the Map Locations of Selected Place Records option to avoid refreshing already geocoded Place records?

In Tools > Preferences > Map Window did you tick Block Refresh of Non-tentative Geocodes?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

Mike - thanks for the reminder.

I have blocked the refresh of non-tentative geocodes in the tools setting, but I must admit I had not ensured that only those places with no geocodes were being checked. I'll try that tomorrow.
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

The point is that in the Map Window only Places with a tick on a white backround are Non-tentative.
Those with a tick on a reddish background are Tentative, as only part of the Place name has been geocoded.
So I think they will get refreshed each time unless you exclude them or manually change their status to Non-tentative.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

Mike - thanks so much for your help so far!

I plan to run the geocoding today on a set of places, but I seem to be missing an important piece of understanding. Apologies if this is this is a tiresomely basic question! I should let you know that I have read the relevant section in the book Getting the Most from Family Historian 6, and can create a map with markers for people in the same project.

You suggest "Instead of using Map All Places, open the Places tab mentioned above, then select a subset of Place records and use the Map Locations of Selected Place Records option."

My issue is in selecting a subset of Place records. When I open the Places tab I have my list of 5,413 places, the first 1,165(*) of which are correctly geocoded, with no pink background to any ticks. I thought I could select, say, the next 200 places which are not coded, simply by clicking on record number 1,166, pressing shift, and clicking on record 1,365 to select ~200 records, shaded blue. At that point, I thought there would be a simple way to name and save that set, ready to be used in the mapping programme. However, I can't find how to select, save, and name that set in such a way that it available for coding. Can you help?

(*) While attmpting various methods, geocoding kicked in and I now have another 300 places geocoded, so now 1,464 of my places are correctly coded. Maybe all I need is patience....
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Geocoding stopped - almost all the time

Post by Jane »

Personally, I would just leave it a few days and you should find over time all your locations will geo-code, or use Mike's plugin to do some of them using Google.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
LornaCraig
Megastar
Posts: 3190
Joined: 11 Jan 2005 17:36
Family Historian: V7
Location: Oxfordshire, UK

Re: Geocoding stopped - almost all the time

Post by LornaCraig »

I thought I could select, say, the next 200 places which are not coded, simply by clicking on record number 1,166, pressing shift, and clicking on record 1,365 to select ~200 records, shaded blue. At that point, I thought there would be a simple way to name and save that set...
You don't need to save the set. Once you have selected the set of records and they are highlighted in blue, just click the Map icon in the main toolbar again and choose Map Locations of Selected Place Records.

By the way, you can select the records either in the list on the left of the Map Window (which I think you are doing, as you refer to ticks with pink backgrounds) or in the Places tab of the Records Window (which is the one Mike was referring to).
Lorna
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

Having selected the Place records in the Records Window (which you have correctly described) then click the red pin Map Window icon in the toolbar and select Map Locations of Selected Place Records.
Depending on which Records Window tab is open, and what records are selected, the options vary and may be greyed out.

The ticks with white or red backgrounds only apply to the Map Window and not the Records Window.
There is a way of adding a Status column to the Records Window to reveal Tentative status, etc.
Use Lists > Configure Records Window Columns, click <Other...> on left, and > in middle.
Set Heading to Status and Expression to %_PLAC.STAT% then click OK.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

This is an update, in case it helps others.
I decided to follow Jane's advice, "Personally, I would just leave it a few days and you should find over time all your locations will geo-code..."
I find that I seem to be able to run a batch of records about every 24 hours, and on each occasion, either ~200 or ~300 records are coded.
After about ten days, I am about half-way through my 5,400 records. I am pretty happy with this pace - as all genealogists have to learn to be patient.
Thanks for all your help!
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Geocoding stopped - almost all the time

Post by tatewise »

Sadie, it ought to geocode more than a few hundred per day. It should be doing thousands.
Have you tried the technique I suggested to select just the Place records that need geocoding and thus avoid reviewing all those that have already been geocoded.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Sadie
Gold
Posts: 13
Joined: 23 Feb 2005 22:45
Family Historian: V6.2
Location: United Kingdom

Re: Geocoding stopped - almost all the time

Post by Sadie »

Thanks for the reminder, Mike! I'll try that again tomorrow.
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Geocoding stopped - almost all the time

Post by Jane »

Mike, my experiments suggest the first day you get a large allocation, then it recovers back some each day, by the sound of it 200-300, so Sadie is doing right, if she left it for a week, she would get 200-300 times the number of days unused.

This would not affect users who were entering data, only where people are geo-coding very large databases for the first time.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
Post Reply