Family Historian User Group

Have a look at https://pluginstore.family-historian.co ... lace-parts, which may help depending on what exactly you need to do. It's one of Mike Tate's plugins, so he may be able to give you tailored advice when he sees this.

There appears to be a great deal of cleaning up that needs to be done.

In 0002, all those Part 1 (http... etc) URL are not valid Place or Address values.
Please explain what they represent and what you hope to do with them.

In 0001, there are Part 1 values such as Coal Miner are occupations that are invalid Place values.
Likewise, all the military companies are not valid Place names. I also see Deceased and an Email address in there.
They all need to be resolved somehow.

Once you have decided what to do with them, it may need a custom plugin to adjust the data.

I don't see anything that the Rearrange Address and Place Parts plugin could help with.

In 0002, all those Part 1 (http... etc) URL are not valid Place or Address values.
Please explain what they represent and what you hope to do with them.

I checked in TMG, and these were all references within source citations. They tended to be birth or death citations of sources which were located at a specific URL. For instance, one was a reference to a source called "John Street Pedigree Chart," and that chart was located at a specific URL, which was cited in the source, which also contained media, which was a copy of the source document. FH's import routine, for some reason, treated this as a Place.

In 0001, there are Part 1 values such as Coal Miner are occupations that are invalid Place values.
Likewise, all the military companies are not valid Place names. I also see Deceased and an Email address in there.

I think that occupations, deceased, and email address will be easy enough to clean up manually. The military company listing here come from the Milit-Beg and Milit-End tags. I'll paste in an FH Facts window so you can see what I'm talking about. Again, it looks like the FH import routine created the Place entry from the TMG citation of the source of this information which was an online source: civilwardata.com. I'll also attach a copy of the relevant TMG page so you can see what these entries looked like in TMG.

Let me know what additional information you need. Thanks very much for your offer to help!

-- Carl

cjdenbow wrote: ↑

08 Jun 2022 04:16

I checked in TMG, and these were all references within source citations.....

Linda Reinfeld described the same problem in her post, when she said:

FH regards locations in source documents as locations, i.e. if a source element comes from the location group FH treats it just like a place in a tag. It's rather annoying - I have decided just to ignore these. That is where I have a lot of urls.

This looks like a problem CP should be made aware of.

The reason FH imports those data values as Place parts is because that is what they are defined as in TMG.
Carl, in your TMG screenshot, in the Tag Entry it clearly says U.S. Standard Place and all the fields below are TMG Place parts.
Check the TMG Help for Field Labels on the Tag Entry Screen where it lists the Place fields as:

There are ten place fields with the following default labels:
Label 1: Addressee
Label 2: Detail
Label 3: City
Label 4: County
Label 5: State
Label 6: Country
Label 7: Postal
Label 8: Phone
Label 9: LatLong
Label 10: Temple

Those are the ten fields shown in your screenshot, so it makes perfect sense for FH to import them as Place parts.

If those fields contain non-Place values then there is not much FH can do about it.

Those are nothing to do with Citations. They are the way TMG displays Fact Tag details including Date on the left and Standard Place on the right, and Memo (Notes) below left (rather like the FH Facts tab). Any source Citations are listed in the box at the very bottom.

It would be useful to see where those URL imported as Place part 1 appear in TMG.

I’m not familiar with TMG but Linda talks about “locations in Source documents” and in Carl’s case he says a URL was “Cited in the Source”. I don’t know whether these are fields in the TMG Source itself or in a Citation (equivalent to “where within” in FH) but it sounds as if an import to FH ought to recognise that NO data in either a source or a citation is a geographical “place” and should simply copy the text.

When referring to the non-URL non-Place values like Coal Miner and military companies, Carl said:

Again, it looks like the FH import routine created the Place entry from the TMG citation of the source of this information...

But his screenshot illustrating such values only shows Fact Tag Standard Place fields (similar to FH Facts tab Place fields).
i.e. They are NOT citation or source information. They are simply the Place where the Fact occurred.
I wonder if Carl and Linda may have misunderstood what those TMG Standard Place fields represent?

tatewise wrote: ↑

08 Jun 2022 12:05

... I wonder if Carl and Linda may have misunderstood what those TMG Standard Place fields represent?

Or there's simply a blurring of what the terms "citation" and "source" actually refer to in these contexts. Goodness knows I've been accused in the past of mis-using the word "citation"...

LornaCraig wrote: ↑

08 Jun 2022 11:45

... it sounds as if an import to FH ought to recognise that NO data in either a source or a citation is a geographical “place” and should simply copy the text.

Actually, among the more obscure tags within a Source-Record format are a DATE and a SOURCE_JURISDICTION_PLACE, not to mention a RESPONSIBLE_AGENCY...

As far as I can see, TMG does not support SOURCE_JURISDICTION_PLACE or any other rarely used Source record fields.

The TMG dialogues make it very clear where Source Citations exist by using the word citation.

To help us understand these data import 'anomalies' it would be useful to see screenshots of the TMG data from which those imports are derived, otherwise it is all just pointless conjecture.

Mike: I'm not sure what you mean by "screenshots of data" in regard to TMG. If you'll explain exactly what you want, I'll be happy to supply it. Thanks for your help. -- Carl

TMG has source templates. The elements used in the source templates are divided into groups: Author, comments, file reference, location, etc. FH has treated the location source element within a source as a valid location when doing a direct conversion from TMG.

Mike referred to the 10 fields in the place location of tags (facts). The problem in FH is not related to tags, but to the location element within the source template.

Lorna is correct - CP should be able to fix this for future migrations.
I don't know how to fix the problem once you have migrated....as I said before, I have decided to ignore it. I am not sure it is a problem, other than the fact that the place records list looks incorrect. Most of the problems are at the end of the list anway...

Linda

Carl, what I would like to see is the TMG version of the URL that import to your screenshot 0002, Part 1 (http... etc) URL.
i.e. Similar screenshots to the TMG and FH ones you already provided for military companies, etc.
Unless you can simply confirm that those URL appear in the Standard Place fields of TMG, so get imported by the same mechanism as Place part fields.

Linda, that is the first time that Source Templates have been mentioned in this context.
I can see that if FH decides the TMG template Location fields have the data type of Place then they will import as Place fields.
Again screenshots of the problem data would help.
What data type should the Location field definition use? Should it be Text or what?
With knowledge of the problem details then it can be fixed by either changing the TMG definition and importing the data again, OR by changing the FH definition and migrating the data using a plugin (if necessary).

Mike says:

Carl, what I would like to see is the TMG version of the URL that import to your screenshot 0002, Part 1 (http... etc) URL.
i.e. Similar screenshots to the TMG and FH ones you already provided for military companies, etc.
Unless you can simply confirm that those URL appear in the Standard Place fields of TMG, so get imported by the same mechanism as Place part fields.

Mike, perhaps, this would answer your question. Here's a screenshot of a section of the TMG Master Place List. It does show that the military company, and regiment information is listed there. However, I can confirm that those URLs from screenshot 0002 do not appear anywhere in the TMG Master Place List.

If you need more information, please just ask.

Carl

Mike-

Sources in TMG have source templates with different fields called source elements. Just like FH has source templates. Unfortunately in TMG the source elements are divided into groups. One of these groups is identified as location. The various types of location can be Address, Author Address, compiler address, informant address, jurisdiction, listserve, location. And you can make up your own source element and put it in the location group. That is how 'url' can be a location element in a source in TMG.

In FH the source element group of location is treated in FH as a place. It should really be a text element. It does not belong as a place element.

Attached are 2 screen shots. The first shows the template digital printed list/index with the source element 'url' which I have created in the location group. The second shows the source. I hope this explains things.

As Carl said, these places/locations do not show in the TMG place list and are not treated as locations/places.

Linda

Linda, thank you for that explanation, which now makes everything clear.

Carl, I suspect your URL 'Place names' are explained by Linda's posting regarding Location elements of Source Templates.
They are imported as FH Source Template fields with the type Place, so the data is treated (mistakenly) as a Place name.

Linda and Carl:
I've experimented in FH and simply changed the data type of such Source Template fields from Place to Text or URL.
Despite the dire warnings, no data was lost and the data fields became plain text or hyperlinks instead of Place names.
Subsequently, all the now unused Place records can all be deleted.

Clearly, changing the data type to Text is valid for all data values. Changing to URL will only work if all the data values in the associated fields are actually URL and not a mixture of data types. That depends on how consistently the data has been entered into TMG.

Does this offer a workable interim solution?
Do you need detailed instructions to edit & sync the field data type?

Make backups &/or work in a copy project before experimenting with this solution.
Note that Source Template changes are not included in Project Backups so must be exported or backed up separately.

The alternative is to change the TMG field element definitions from 'Location' to some other group and import again.

Hi Mike -

I changed the data element/Field Definition (URL) in the source template (online study project) to 'text'. Easy.

However, when I went to the source record that uses that source template the Source record shows 'Undefined Template Field'. But it still has the data in it.

Then when I look at that source property box there is no data in the field labeled URL

And then in reports, etc I loose the data.

I am probably missing something, because you did say:
Do you need detailed instructions to edit & sync the field data type?
I didn't sync anything. Also I didn't even get any dire warnings.

I took screen shots of everything in case that would help.

I like your suggestion enormously - the source elements in TMG should not be labeled as 'location'. I would change the data in TMG ( I have already changed thousands of pieces of data in TMG to accomodate FH - and to make my data more consistent) but I have over 2600 sources that use this element. Hopefully I've just missed a sync step.

Thanks for your help. As always, I have learned a lot more about FH and can move around more easily in the system.

Linda

Linda, thanks for your response re: templates and locations. That explained the TMG situation very clearly.

Mike: Yes, I would like detailed instructions on making these changes. I'm very much a newbie to FH, and I don't know my way around the program all that well. On the TMG side, I was a beta tester back in the DOS days so I'm pretty proficient in its use.

Carl

Remember that the only Template Field definitions that need changing are those that have the data type Place and derived from the TMG Location group. Initially, it is only necessary for those Fields that have imported data from TMG.

Ensure you have suitable backups of both the Project and the Source Template Definitions.

First, use the Tools > Source Template Definitions... command to open the Source Template Definitions dialog shown in the FH Help page Source Template Definitions Dialog. I suggest you explore that Help page and the pages it links to in order to understand the editing processes.

In FH the Source Template Definitions exist outside Projects in the global settings similar to Queries and Fact Sets.
Whereas, most global settings automatically sync/update every Project, the Source Template Definitions must be synchronised with the Source Template Records within Projects.
It is feasible to update either the Definitions first or the Records first and propagate the changes in either direction.

The Templates that need changing will probably exist in the Collection called TMG Import.
Select a Template to change and click the View/Edit... button.
Select a Field in the lower pane that has a Type of Place, click Edit, change the Type to Text or URL, and click OK.
Now click Save and Close top right (or to change the next Template click Next).

Back in the Source Template Definitions dialog, you must propagate those changes to the Source Template records in this Project. If you have similar Source Template records in other Projects then they must be updated similarly.

Click the Compare/Sync with Source Template Record... button. (Linda, this is the step you overlooked.)
The Field changes that need a sync are marked with a red X on the right.
Select one and click View/Sync... button below, to see the Type that is being synchronised.
Click Perform Task > Update record field (right-side) to match definition field (lef-side)... and OK warning.
Close both dialogues and that should have completed the change.

After checking the changes, the now unused Place records can be deleted as explained in Delete One or Many Records.

Hi Mike -

When I click on tools/source template definitions the only collections I see are All, Advanced, Custom (4 of these where I was testing/fooling around with the templates before they were imported from TMG) and Essentials. There are no TMG templates there. The TMG templates are shown in the record list as a tab - Source Templates.

In the steps I outlined above, the first step I did was to change the template in the records window.

Linda

Linda, I think you will need to run the Create Source Template Definitions plugin first, to create the definitions from your imported TMG set. The question was raised in this topic: Location of imported template collection? (20268)

Yes, sorry, I overlooked that prerequisite. It probably applies to both Linda and Carl.

Linda, before running the plugin suggested by Lorna, I advise you undo the change made to the Source Template Record.
i.e. In the Field Definition: Url revert the Type back to Place so the settings are as imported from TMG

BTW: Since they would appear to hold URL, why did you choose to change to Type: Text instead of Type: URL?

Thanks, Mike. I will attend to this on Monday. I will be out of town this weekend. FYI: I had previously run the Create Source Template Definitions plugin. -- Carl

Mike -

Thanks for the detailed instructions. And also thanks to Lorna for the plugin 'Create Source Templates Definitions'. I didn't know you had to run that as one of the steps in conversion. My TMG templates seemed fine, I could use them with sources, and didn't even realize something was wrong until I tried to change the type.

It took a while, because of a difference between TMG & FH - the plugin wouldn't run at first because it said I had duplicate source template names. In TMG if there is 'extra' data in a source there is no problem. The source will output only the data that is in the templates. Sometimes when you change source templates for a source you end up with extra data. That is what the problem was. FH called them different source templates with the same source template name. So I had to fix that problem first.

I had 16 source templates with the element 'url' that I changed from location to text. I chose text because that should cause the least problems. The source field is not used as a true url either in reports or online, so I thought text would give me the least problems.

I also tested the change with GedSite, my online web builder, and there were no problems.

Thank you for the help

Linda

I believe FH is different from TMG regarding Source Templates.
In TMG, there is only one set of Source Template definitions.
In FH, the Source Template Definitions exist with the other global custom settings outside Projects, and are represented within each Project as required by Source Template Records.
So within a Project, the Source Template Records are typically a subset of the Source Template Definitions.
The Source Template Definitions are synchronised with the Source Template Records using the tools discussed earlier.

What I don't understand is why the TMG ( and RootsMagic) direct import process only imports the Source Template Records and does not also create the matching global Source Template Definitions. In my opinion, it should not be necessary to use a Plugin to perform the task of creating the global Source Template Definitions.

Has CP explained why the direct import process does not create global Source Template Definitions?

In their Importing from The Master Genealogist guide, near the end of Source Templates, it says: If you already have a Source Template record within a project, you have no need of a source template definition. However, this thread shows that is not a valid assumption and several users find the missing definitions awkward. The Plugin should not be needed!