Family Historian User Group

ColeValleyGirl wrote: ↑

20 May 2023 08:54

On a more radical note, I could be tempted to replace all references to Method 1 and Method 2 with 'Splitting' and 'Lumping' -- partly because I can never remember which is which and 'Splitting' and 'Lumping' are more obvious descriptors. (Not in the Ancestral Sources articles, of course, where 'Method 1' and 'Method 2' originated.) I'd change the title of the article that started all this to: "How Many Sources Do I Need?", because newcomers may not recognise from the current title what it is talking about.

I think this makes sense, but perhaps the first time these are explained/used in the article add a comment such as "referred to as 'Method 1' by some Family Historian users"

ColeValleyGirl wrote: ↑

20 May 2023 08:54

... I think we should avoid the word 'data' (us old warhorses know what it means but a new genealogist may not think of what they're working with as 'data'). ...

Sounds fine to me except I'm slightly concerned by the reference to "data" here. I wrote

the output Citation = Data from the Source-record plus optional, additional data held within the citation-data.

I'm still trying to resolve any potential confusion about where information can go. How do we make it clear that in lumping, we're talking about putting specific information for the citation in the bit below the line and not above the line in the Source-record?

Is it something like:

the output Citation = Information from the Source-record plus optional, additional information unique to that particular citation.

AdrianBruce wrote: ↑

20 May 2023 10:37

ColeValleyGirl wrote: ↑

20 May 2023 08:54

... I think we should avoid the word 'data' (us old warhorses know what it means but a new genealogist may not think of what they're working with as 'data'). ...

Sounds fine to me except I'm slightly concerned by the reference to "data" here. I wrote

the output Citation = Data from the Source-record plus optional, additional data held within the citation-data.

I'm still trying to resolve any potential confusion about where information can go. How do we make it clear that in lumping, we're talking about putting specific information for the citation in the bit below the line and not above the line in the Source-record?

Is it something like:

the output Citation = Information from the Source-record plus optional, additional information unique to that particular citation.

I almost suggested Information but thought it might clash with ESM's usage of the word; maybe that isn't important. Or what about Detail?

NickWalker wrote: ↑

20 May 2023 10:34

I think this makes sense, but perhaps the first time these are explained/used in the article add a comment such as "referred to as 'Method 1' by some Family Historian users"

OK.

ColeValleyGirl wrote: ↑

20 May 2023 10:44

AdrianBruce wrote: ↑

20 May 2023 10:37

... Is it something like:

the output Citation = Information from the Source-record plus optional, additional information unique to that particular citation.

I almost suggested Information but thought it might clash with ESM's usage of the word; maybe that isn't important. Or what about Detail?

I think "Information" is generic enough and I doubt many devotees of ESM will feel confused here.

"Detail" doesn't quite fit in my mind - "Detail from the Source-record" doesn't seem to read right as a starting sentence. Detailed what? I'm tempted to ask.

I think that "... plus optional, additional detail unique to that particular citation" sounds OK but I'd rather not use two different words. Either both should be "detail" or both "information" and since I feel the first phrase has to be "Information", then both should be "information".

I think one problem with the existing KB article on Method1/Splitting and Method 2/Lumping is that, as it says, the examples “are based on using ƒh6 or ƒh7 with Generic Sources, but the principles apply to all versions of ƒh and all kinds of Sources.” What it doesn’t say is that before V7, when all sources were generic, the user had to make their own decisions about how much lumping to do for different generic source types.

It’s different now. A new user starting out with FH7 (not migrating from another product) can use the program ‘out of the box’ without ever thinking about the issue at all. This is because CP have provided the Essentials collection of templates. The majority of these have been designed for Splitter sources, hence the description “One source per registration certificate…” or “One source record per census return (census household)….” etc. but the Civil Registration Index template is “One source record per index…” so it’s a Lumped source. The description of each template explains how it is intended to be used. If the new user adopts the Essentials templates and uses them as designed they never need to make a lumper/splitter decision because CP have made the decision for them.

I think the article in the KB should start off by making this clear, because I’m pretty sure the vast majority of new users just want to know how to use FH out of the box.

tatewise wrote: ↑

20 May 2023 09:17

I'll go along with all those proposals.
In particular "splitting" and "lumping" concur with the Legacy definition and avoid the implication that "splitter" and "lumper" apply to the user rather than the source.

Just so long as the reader knows that any source can be entered as either a lumped source or as a split source! An example of the same source being entered lumped and split would be helpful! Because as I’ve pointed out in the past and Harvard University says:

Lumpers and Splitters: For several centuries taxonomists of the natural world have been divided into lumpers, who seek to merge a larger number of proposed species or genera into a smaller number, and splitters, who seek to move in the opposite direction.

This has been a discussion in taxonomy for many years. Wikipedia say:

Lumpers and splitters are opposing factions in any discipline that has to place individual examples into rigorously defined categories. The lumper–splitter problem occurs when there is the desire to create classifications and assign examples to them, for example schools of literature, biological taxa and so on. A "lumper" is a person who assigns examples broadly, assuming that differences are not as important as signature similarities. A "splitter" is one who makes precise definitions, and creates new categories to classify samples that differ in key ways.

The Source_Record is a classification of a source, the Source_Detail (Where in Source) describes the class member!

LornaCraig wrote: ↑

20 May 2023 13:02

I think one problem with the existing KB article on Method1/Splitting and Method 2/Lumping is that, as it says, the examples “are based on using ƒh6 or ƒh7 with Generic Sources, but the principles apply to all versions of ƒh and all kinds of Sources.” What it doesn’t say is that before V7, when all sources were generic, the user had to make their own decisions about how much lumping to do for different generic source types.

It’s different now. A new user starting out with FH7 (not migrating from another product) can use the program ‘out of the box’ without ever thinking about the issue at all. This is because CP have provided the Essentials collection of templates. The majority of these have been designed for Splitter sources, hence the description “One source per registration certificate…” or “One source record per census return (census household)….” etc. but the Civil Registration Index template is “One source record per index…” so it’s a Lumped source. The description of each template explains how it is intended to be used. If the new user adopts the Essentials templates and uses them as designed they never need to make a lumper/splitter decision because CP have made the decision for them.

I think the article in the KB should start off by making this clear, because I’m pretty sure the vast majority of new users just want to know how to use FH out of the box.

I did think about proposing we ditched the article all together -- as Lorna says, it isn't a decision that most newcomers to the product will need to make. New genealogists (who haven't used another product) can adopt the Essentials templates; and FH now supports lumping much better, so migrants from other products can in the main stick with the way they've previously worked (except for the known Bibliography constraint).

KFN wrote: ↑

20 May 2023 13:06

This has been a discussion in taxonomy for many years. Wikipedia say:

Lumpers and splitters are opposing factions in any discipline that has to place individual examples into rigorously defined categories. The lumper–splitter problem occurs when there is the desire to create classifications and assign examples to them, for example schools of literature, biological taxa and so on. A "lumper" is a person who assigns examples broadly, assuming that differences are not as important as signature similarities. A "splitter" is one who makes precise definitions, and creates new categories to classify samples that differ in key ways.

I'm not convinced 'opposing factions' is relevant here, where individuals making decisions about how to structure their sources aren't affecting anybody else.

ColeValleyGirl wrote: ↑

20 May 2023 13:10

I did think about proposing we ditched the article all together -- as Lorna says, it isn't a decision that most newcomers to the product will need to make....

I had thought the same (ditching it all together) but hesitated to make such a radical suggestion!

ColeValleyGirl wrote: ↑

20 May 2023 13:15

KFN wrote: ↑

20 May 2023 13:06

This has been a discussion in taxonomy for many years. Wikipedia say:

Lumpers and splitters are opposing factions in any discipline that has to place individual examples into rigorously defined categories. The lumper–splitter problem occurs when there is the desire to create classifications and assign examples to them, for example schools of literature, biological taxa and so on. A "lumper" is a person who assigns examples broadly, assuming that differences are not as important as signature similarities. A "splitter" is one who makes precise definitions, and creates new categories to classify samples that differ in key ways.

I'm not convinced 'opposing factions' is relevant here, where individuals making decisions about how to structure their sources aren't affecting anybody else.

I agree that “opposing factions” is too harsh to lay at the feet of a newbie! The point this statement makes is that some people see a source in a different way than others an “opposing view”! (Edit: Differing viewpoints)

An example would be a source could be a website that contains all kinds of varied data, obits, births, marriages, etc. A lumper would just say the source is that website and detail each piece of data is the detail, while a splitter could take each page of the website call it a source and have very little or no detail. A book can be a lumped source, but a split source can start with a page in that book!

LornaCraig wrote: ↑

20 May 2023 13:25

ColeValleyGirl wrote: ↑

20 May 2023 13:10

I did think about proposing we ditched the article all together -- as Lorna says, it isn't a decision that most newcomers to the product will need to make....

I had thought the same (ditching it all together) but hesitated to make such a radical suggestion!

I'm not sure it's true to say that no-one makes the Method 1 v Method 2 decision any more. Yes, total newbies will probably take the path of least resistance and follow the templates, but they aren't the only newcomers to FH. Some people (incomers or continuing users) embark on source re-dos - or at least, contemplate it. One aspect might be converting to templated sources but another might be to question whether a degree of splitting or lumping of at least some source-types might be beneficial. They might welcome an article that talked about the consequences of each method.

That's not to say the article is aimed quite right. It has, e.g., "Number of Citations to each Source Record - [Method 1] is the best method if you expect to have multiple similar Citations to a Source, each with little or no data in the Citation." With respect, that's the wrong way round. You don't decide on your number of similar citations first and choose a Method accordingly - you make the choice for other reasons, and "multiple similar Citations" (or whatever) is the consequence. But let's leave that bit for later...

I was wondering about recasting it not as a comparison but as a set of considerations when deciding whether to split or lump an individual source/source type.

If my recollection is correct, that is how it was presented in the old KB. The comparison table was introduced in the new KB.

ColeValleyGirl wrote: ↑

20 May 2023 19:20

I was wondering about recasting it not as a comparison but as a set of considerations when deciding whether to split or lump an individual source/source type.

I think that's what it should be - some of those considerations will surely end up as comparisons, quite sensibly. The considerations angle probably leads neatly in to examples of some "sources" that the vast majority of people lump (e.g. BMD indexes) even when they are normally splitters. Similarly virtually no-one would split a book - unless it's a book with internal parts that have little to do with one another other than some over-arching theme (e.g. a compendium of "stuff") when the decision could go either way.

Could this also then be the place to mention that different people will split their sources in different ways? E.g. a single source-record for the 1880 Census as a whole or a source-record for each county in the 1880 Census? Or would you split the 1939 from Ancestry from the 1939 from FindmyPast - which is a different dimension, as it were? Probably no point in going to town on the different possibilities but they ought to be mentioned somewhere.

AdrianBruce wrote: ↑

20 May 2023 20:28

Could this also then be the place to mention that different people will split their sources in different ways? E.g. a single source-record for the 1880 Census as a whole or a source-record for each county in the 1880 Census? Or would you split the 1939 from Ancestry from the 1939 from FindmyPast - which is a different dimension, as it were? Probably no point in going to town on the different possibilities but they ought to be mentioned somewhere.

I've talked to people that will take two different censuses and add them as source_records differently. I'm not in favor of this approach, but to each his own.

Example:

1900 US Census as a Source_Record, with multiple citation entries. They do this because they have multiple family members that moved to various parts of the US, and would find each person in different Enumeration District. They would store detail about the census in general on the Source_Record. The "Citation" would contain specifics about the Enumeration District (State, County, District, Page, Line #) with an image of the page.

vs.

1880 they would only have one relative in the US and they would generate a single Source_Record that has the Title which includes, all of the information about the 1880 Census and the one Enumeration District that person was found and no detail. Everything they needed for the Source_Record.Title came directly from A.com.

In cases like this the data is stored differently not by "Type" (Census), but for convenience of entry.

AdrianBruce wrote: ↑

20 May 2023 19:12

I'm not sure it's true to say that no-one makes the Method 1 v Method 2 decision any more.....

I don't think anyone has said that. The point is that before v7 everyone had to make that decision themselves, for each generic source type. But new users now don't have to make those decisions if they simply adopt the Essentials templates. I think the article needs to say that, so that new users understand that each template has already been designed to be used in a certain way (mostly 'split', but the one for an index is a 'lumped'). They just need to read the descripiton of each one to see whether it's "one source per (e.g.) household..." or "one source per index..."

The article can then go on to explain that there are other ways of doing things, either using generic sources or by designing one's own templates. That's when the pros and cons of lumping and splitting need to be understood.

LornaCraig wrote: ↑

20 May 2023 21:53

AdrianBruce wrote: ↑

20 May 2023 19:12

I'm not sure it's true to say that no-one makes the Method 1 v Method 2 decision any more.....

I don't think anyone has said that. ...

Sorry - my mind over-simplified your logic and the direction it was (or wasn't) going in.

AdrianBruce wrote: ↑

19 May 2023 19:39

OK - sticking my head into the lion's jaw ...

Following on from the above, here's my revised attempt at a definition of Citation, attempting to take on board some of the later comments. I may have missed some of those comments or been able to work out a better form of words. (Note I'm still not happy about the wording of "fact, etc."). As I said before, there is a deliberate attempt to foreshadow Method 1 & 2 / Splitting & Lumping within these words, else some phrases might not be necessary. I also tried to pick up on some words in the FH Help Glossary.
------------------
In Genealogy, external to FH:
A citation is linked to an item of data. It identifies a source that provides evidence for that item of data, and therefore links the item of data and the source. It can also provide additional qualifying information about the source, and about how the source relates to the item of data. Externally to FH, it often appears as a footnote, endnote, or other form of reference.

Within FH:
A citation (aka source citation) is linked to a "fact, etc." (See below). It identifies, and links to, a source-record that provides evidence for that "fact, etc.". Information from the (real world) source can be held either on the source-record or within the citation-data itself. Information about how the (real world) source relates to the "fact, etc." can be held within the citation-data itself.

The citation is unique to that combination of "fact, etc." and source-record. Other citations may have the same source-record and additional information but there is no means of linking them in FH.

In FH, the output Citation in a report etc. = Information from the Source-record plus optional, additional information from that particular citation. The output Citation can appear as a footnote, endnote, or other form of reference.

----------------

The Glossary puts the definition under Source Citation - I prefer to put it under Citation as that's more likely to be the term searched for but so long as there's a link, I'm not that fussed - except that the word used throughout my definition is "citation".

I put the term "fact, etc." in quotes to cover events, attributes, names, witnesses, individuals, families, places, media, notes, etc, etc - whatever can have sources cited against them. The word "fact" should not be thought to imply any absolute certainty or finality. The FH Glossary uses "an item of data" rather than "fact, etc.".

The "optional, additional information from that particular citation" should be understood to cover not just templated items held at the citation level but also things like "where within" for generic sources and assessments for both. I believe it important to explicitly separate this from the link elements of "citation" that point to the "source record", etc.

Not sure quite where these should go (if anywhere) but I decided to compare a split and a lumped census Source Template Definitions, along with corresponding citations.

The first image shows the STD out of the Essentials collection for a census return. This is split, i.e. it's designed to set up source records as one per household. I think the only difference from the Essentials template is the template name. The second image shows the same template after cloning under a different name and adjusting to be a lumped design. In this case, it lumps up to a source record per census-year-and-collection. This means that the 1881 census for England & Wales (say) might have 3 source records - 1881 Ancestry, 1881 FindMyPast and 1881 LDS transcription, say. Individual households would then appear on citations "beneath" the appropriate source-record. Fundamentally, all that has happened is 4 fields have been marked as citation level compared to the split STD, with the consequent need to take those items out of the (source) Record Title Format. Further shows to follow...

(this follows on from my previous post)
These 2 images leap forward to show Citation Windows for each of the two previous source templates - one citation to the split template, and a second for the lumped per year / collection. In order to provide the closest comparison I have made up a source-record and citation for the same household under both templates, i.e. it's the same data presented in both fashions. Nobody would ever do that in real life, of course, unless it was to investigate the differences. The first image shows the household using the split template. (If anyone wants to debate whether the Repository is meant to be TNA or Ancestry, please save that for another day! It seems to me that both are needed.)
The above shows the same original data but using the lumped template. Notice how the 4 template items have moved from the source record part to the citation-specific part "below the line".

Note also that (unless I've missed something) both split and lumped template sources produce identical Footnote, Short Footnote and Bibliography entries.

Does this help?

That is a handy summary.
One reason why splitting is recommended is that the lumped Citation-specific details get replicated in the multitude of Citations that are needed for all the Census household facts. That replication is frowned upon by database purists and makes correcting any of those Citation details a bit more difficult. The split option is easier to manage.
Lumping also makes identifying the Census household a bit more difficult because the multitude of identical Citations must be isolated, whereas the split alternative is simply all the Citations linked to the Source record.

I noticed the {collection} field is not included in either example, is that by design?

Adrian,

Can I ask, without getting in trouble, what do the two lump vs split GEDCOM files look like?

I’d want to know how they would translate into my own database.

Thanks

cwhermann wrote: ↑

25 May 2023 00:05

I noticed the {collection} field is not included in either example, is that by design?

i.e. - it's not used in the output footnotes or bibliographies...

Err. Good question. It's not used in the screenshots because it's not used in the original "Essential" template and I deliberately wanted to take that untouched to avoid any debate about me loading the dice in any comparison between Splitting and Lumping. (And, as has been suggested in the past, many will take the Essentials templates untouched anyway.)

But yes, it would make more sense to include the collection in the output footnotes or bibliographies. Just as it would make sense to resolve whether that Repository should be where the source is (Ancestry) or where the source of the source is (TNA, Kew). I suspect both are needed somewhere.