Family Historian User Group

I am attempting to import a gedcom from RM with a lot of difficulty. I have tried exporting RM7 and RM8 with the same results. The first file, I imported as a gedcom. It imported without too much difficulty until I looked at the Log File. There were approximately 2000 exclusions and twice that many Info entries. The citations did not show up at all in the Records view, but they they do show up from the Property Source window, but they do not look quite like they should. I have about 400 citations using a custom source template. Everything was working very well in RM prior to the exports. I also tried importing the file using the option of importing from other family tree file, which went a lot faster, but it did not generate a log, so I don't know how well it did other than the citation do not show up at all in the Records view, but they look perfect when viewed from the Property Source window.
To sum it up:
RM7 gedcom to FH gedcom = Report with problems & citations incorrect.
RM7 gedcom to FH other tree = No Report, so problems?? But citations correct.
Any Idea's?

Ah, clear now - see my comment in the templates thread.

RM omits a lot of data in its GEDCOM export, so what comes into FH always has a lot of gaps, exceptions and undefined fields. The template and to-do list plugins referred to in the KB are a sticking plaster to fix a couple of the more significant omissions.

However, the good news is that FH can now import the RM7/8 database file directly (providing that you are using version 7.0.8.2 or later), and this works extremely well.

Try again with a direct import. It should be self-explanatory, but post again if anything is not clear.

I guess I was not very clear in my description. I did the direct import and what is being interpreted (excuse my lack of proper terminology) is what I get on my end, is as what I described as the RM7 gedcom to gedcom - as in, the log gives me many errors and the citations come in some-what scrambled. The import I did using "Importing from other tree file" came in with correct citatations, but did not include a Log File to see what is Excluded/messed-up. I want to know what data in incorrect so I can fix it, but the direct import gave a very bad import. Should I stick with the import I did using the "Importing from other tree file" - even though it does not give me a report?
It's late, and I need to get some sleep.
I appreciate any thoughts/suggestions... tomorrow

Thanks Chris - it was even later here, so I can now read your posts and interpret the images more thoroughly .

First, some terminology, which you will obviously know already, but here for completeness. A Source Template defines the overall structure of a source. A Source is more specific, and is a specific example using that template. Citations are the links between a Source and the specific Events that are cited to the source. RM and FH have slightly different definitions of Event, but that is not relevant here (in RM, everything is an Event, but FH splits Facts into Events and Attributes).

In RM, all sources are built on templates. In FH7, templates are optional, and the term "Generic Source" is used for sources with no template. Prior to version 7, all FH sources were generic, so many established users have stuck with that way of doing things.

GEDCOM does not support Source Templates, so apps have to use custom extensions to include the template information in the GEDCOM file. The RM GEDCOM export includes full details of your user-defined template, but only limited detail from standard ones. Although FH has its own custom extensions for source templates, they are different to the RM ones. When FH imports an RM GEDCOM file, it discards all the template information that it does not understand, and converts all sources to generic. Much of the detailed structure of individual sources is relegated to what FH calls "Undefined Fields", which are not displayed in the Records Window unless you expand the plus sign at the left hand edge of the display. The Template field is blank in your last attachment, as the templates have been discarded.

This mangling of sources creates a lot of entries in the import log, as you have seen. The "Importing RootsMagic Source Templates and recreating detailed citations" section in the KB describes use of a plugin that repairs most of this damage.

How do you fix the GEDCOM import? Simple answer, don't. Even the plugin can only do so much, and there do seem to be flaws in the detailed GEDCOM structure that have persisted into RM8.

Concentrate on the direct import, using the "Import from other family tree file" option. It doesn't matter whether you use the RM7 *.rmgc or RM8 *.rmtree option, so use whichever is your master copy. With this option, your source structure is intact. There are some minor differences, such as different criteria for defining the Bibliography field (FH does not permit citation level fields), and some minor interface details of templates, such as the detailed field descriptions, are omitted out of respect for RM IP.

There is no import log from this type of import, as there are no errors to report on.

The Records Window in FH displays only Sources, not Citations. The 'Links' column displays the number of Citations to that source. For example, your Neth Birth Record - Friesland, Achtkarspelen source has 106 citations. Highlight that source, and select View > Citations to Source Record from the top menu to display each individual citation.

Once you are happy that FH has imported your data correctly, you can concentrate on any changes that may be necessary to optimise it in FH, such as adding back any missing template details, redefining bibliographies, and possibly converting the "lumped" sources that RM uses extensively to the "split" sources that are a better fit to FH architecture.

They can be later postings, but have a quick look at Sources and Citations in Version 7 for New Users to get an overview of how FH handles templates, sources, and citations.

Thanks so much for you very thorough response! It is now all making much more sense. It was awfully nice to find all my sources nice and safe in the View > Citations to Source Record! And I must admit that the thousands of exclusions in the log files were rather upsetting. I feel much better - and much more informed! Thanks!

I have always been a die-hard lumper, but yesterday I had made the decision that if I was going to embrace FH, I was going to have to become a splitter - or more accurately, I was going to have to use the supplied templates as pointed out in their description (One source record per event unless it is a list... and a few other situations) and only change the source fields to meet my need to accurately and easily be able to find my way back to any cited fact. I concluded this based on the help article 'Source-Driven Data Entry'. I will then need to modify my existing citations to be a better fit within FH. On that note, I ran the Lumped Source Splitter plugin on one of my citations, but it returned an error and requested that I import my custom source template - which I proceeded to do. I then ran it again and it offered three fields that I could choose from to split it. I picked three (out of 11) and ran it, but even though it said it was successful, I can't tell any difference from the other sources/citations. Was that the way to convert my sources, or if not, can you tell me where to go to find out the best way to convert them? Wait a minute, I just noticed that all of my citation fields, viewed from the Property box (see below) are now converted to my custom RM template. Huh!?! I'm not sure that was the direction I wanted to go, but...??

I think the question to myself is - What direction would I like to go with my sources/citations and then how do I get there?
- They should be configured in the best manner for FH, such that they work best with "the split sources that are a better fit to FH architecture" (your words, not mine).
- I think that should happen by utilising the FH Essentials collection of templates and customising those enough to be able to return to the source of the source while staying within FH rules of operation.
- I need to update my existing data to match new template design before moving forward with my family research. This applies especially to the (I was going to say "Master Source list") but instead I'll say the Records Window, Sources list.

I would like to here your thoughts on this, but I wonder if we should move this over to a different forum group. We've strayed from Import/Export somewhat. If you agree, how would we do that (this is also my first time using a forum)
Thanks Mark.

Feels ok in this forum, but I’m just a humble user. If one of the moderators thinks we’re straying off piste, I’m sure they’ll chip in and tell us!

What Calico Pie call Source Driven Data Entry is a process for extracting data from a defined templated source using a wizard called a Data Entry Assistant. In principle, it could be made to work for either splitting or lumping, but all the existing examples are based on splitting. It was introduced in FH7, and I think it’s fair to say that it hasn’t proven that popular with existing users, but many new users who aren’t so set in their ways have embraced it.

I’m interested in your description of using the Lumped Source Splitter. When you imported your RM file, did all the Source Templates come across? Were all the sources attached to templates? The plugin was designed originally for generic sources, which are easy to split as there is just one citation field. Splitting on up to three template citation fields is an arbitrary choice. It’s been used successfully with standard RM templates, but may not be optimised for all templates. If you have a specific example in mind and how you want to split it, I can advise the best option.

Yes, all the sources came over and the citations were all attached.
You asked, "If you have a specific example in mind and how you want to split it, I can advise the best option". My answer to that is a very vague "I don't know"! All I can say is that from what I have read, it seems I need to split my sources to make them compatible with FH. I think this is to break out the citations so that each citation becomes a source, which is what FH likes. Those are my ignorant assumptions. I think I should be asking you - what should I do to make my existing sources work and behave properly in FH (all my sources came from the same custom template). It is designed to work best with Dutch records from allefriezen.nl. I would like to get away from using this and instead use FH standard templates, but I need to get my existing records look and behave like FH standard sources. Below are two screen-shots of the template structure and how the appear in the Records pane.
As for the Data Entry Assistant. I would love to use the Data Entry Assistant! It sounds like it will work perfectly with my normal workflow. The problem is that the plugins for it are all for the UK and I work in the Netherlands and the US.

Chris,

The simple answer is that you don't need to do anything. The sample project and documentation tend to be written for one particular working style, and the provided Templates and DEAs are strongly biased towards splitting and UK-centric research (just as RM and FTM are US-centric). However, FH is a wonderfully flexible product, and if your particular style worked for you in RM, there is absolutely no reason why you couldn't continue with it in FH.

If you want to move your project to a model based on split sources using templates, you have to ask yourself whether you will keep existing sources as they are and create new ones in a different style, or pause new research for an extended period while you rework all your sources to new templates. If you do the latter, it's potentially a lot of work. Only you can judge whether it is worth it.

Splitting versus lumping is a personal choice. I came to FH from FTM, which manages lumped sources extremely well. For me, the showstopper was that FH has no concept of a "shared citation". To take an extreme example, if you consider a census entry for a large household, with the census cited for census/residence, age, occupation, place of birth, etc, that could be dozens of individual citations. In FTM, they are managed as one entity. In FH, they are all independent of each other - change one, and you have to change all of them. For me, that's so contrary to good database design that I couldn't work with it.

Earlier versions of FH actually made it more difficult to lump than split for more complex sources. FH7 has levelled the playing field somewhat, but it still doesn't manage lumping nearly as well as FTM (even in the old 2012 edition that I used).

You will soon pick up if you haven't done so already that FH hangs its colours very firmly on the GEDCOM mast. All too often, if somebody asks "why doesn't FH do this...?", the stock answer is that "GEDCOM doesn't permit it", no matter how sensible on the surface the option may be. GEDCOM doesn't have shared citations, so FH doesn't either (although it could manage them in its internal plumbing without violating GEDCOM if the developers chose to do so).

Having said that, many FH users don't find that a problem. RM7 doesn't support shared citations either, so it is nothing new for you (but I believe that RM8 does).

On the mechanics of splitting, if you wanted to split the example you showed, I would select Person of Interest, followed by Event Date. That would give a series of new sources with new titles following the format "Lumped Source Title: Person of Interest, Event Date", with each source linked just to one person. However, the new source would still use your existing template. You could create a new template containing all the relevant fields, perhaps modelled on one of the provided examples, and transfer your sources across to it, but that is not a simple one-click process.

I think you have a period of playing and experimenting coming up, whichever way you jump...

Mark1834 wrote: ↑

22 Mar 2022 19:58

What Calico Pie call Source Driven Data Entry is a process for extracting data from a defined templated source using a wizard called a Data Entry Assistant.

Not entirely correct -- a DEA is one possible method of Source-driven data entry, but there are others, as described in the Help File

In principle, it could be made to work for either splitting or lumping, but all the existing examples are based on splitting. It was introduced in FH7, and I think it’s fair to say that it hasn’t proven that popular with existing users, but many new users who aren’t so set in their ways have embraced it.

I think you're referring to DEAs here, not Source-driven Data Entry which many existing users have been using for years, either via AS or using Automatic Source Citation, for example.

Thanks for the clarification Helen. IMO, it doesn’t help that CP make that link in their promotional material for FH7, so imply that DEAs are an integral part of Source-driven Data Entry (their capitalisation), rather than just a new tool for supporting a general style of working that most apps have been compatible with for years.

The key point we are both making is that the shiny new tools are a way of working in FH7, not the way of working.

Thanks for your insight, Mark.
You have given me much to think about, but I think I better understand the pros and cons of the different directions I might go. And now it is time to do a little pondering and playing.
Thanks!