* Sorting Surnames - especially prefixed Surnames

[davidf]

Various threads have pondered issues with surname sorting and prefixes:
Surname prefix (SPFX) -- more generally, handling structured names. (20719)
How to sort results by surname when using Work with Data option? (21197)

Rather than repeat all the above, I want to ponder how we think we should actually use FH (including the Given Name, Surname Prefix and Surname Fields (GIVN, SPFX and SURN - accessed via the All tab), as well as the more "conventional" (?) Name and Slashed Surname constructions (NAME and NAME:SURNAME)

FH conventionally sorts on "surname" (being the stuff between the slashes - NAME:SURNAME) and then the "given names" (being the stuff outside the slashes - both before and after the slashes - NAME:GIVEN_ALL). How this happens for prefixed surnames (Jacqueline du Pré, Vincent Van Gogh, Ursula von de Leyen, Ludvig van Beethoven, Wikus van de Merwe, Ronan O'Gara, etc.) is more problematic; rules swiftly become complex and there always seem to be exceptions.

The issue I am trying to get my mind around is what accountants call "substance over form".

Aside: [In the accounting world there were debates about "off balance sheet financing" - whereby companies could keep their debt levels down (and credit rating up). Instead of borrowing to invest they would lease assets and the lease payments went through the profit and loss account as an expense. Technically the "form" of the transactions was being acknowledged, but the "substance" was that both methods incurred similar forward levels of obligation to make future payments (leasing charges or debt interest and repayments) and these needed to be "properly and consistently" reflected in accounts. Similar thoughts applied to Final Salary Pension schemes. Showing the "substance" of these obligations so frightened companies that many closed their "defined benefit" pension schemes.]

In terms of names, all the above examples can follow "correct form" and can be "chopped" into "Surname Prefix" (du, van, von, von de, van de, O' etc.) and put in the SPFX field and "Surname" put in the SURN field. Mike Tate has written a plug-in that will do this chopping.

But the effect of doing that does not reflect the substance, the realities. We can cite (and argue/debate) various examples but purely as an illustration, I offer (in terms of "name", "sorted as"):

• Ludvig van Beethoven > Beethoven, Ludvig van
• Ursula von de Leyen > von de Leyen, Ursula
• Jacqueline du Pré > du Pré, Jacqueline
• Simone de Beauvoir > Beauvoir, Simone de
• Ulrich Le Pen > Le Pen, Ulrich
• Vincent Van Gogh > Van Gogh, Vincent
• Ronan O'Gara > O'Gara, Ronan

I have even read that in different parts of Belgium (Flemish speaking Flanders or French speaking Wallonia), the order in which names may be indexed can vary not according to the origin of the name but according to writer/compiler's geographic location! Presumably similar considerations may apply in other multi-lingual states such as Switzerland.

The question then is how to reflect the "substance" of names, so that they sort correctly, form correct sentences where we are referring to people by their "surnames" ("... the Beethovens lived in Bonn ..."), and capitalise correctly when required ("von de LEYEN"?).

Elsewhere I have pondered whether we should recognise "John /Smith/", "Jacqueline /du Pré/" as being "secondary sort/primary sort/" (substance) rather than "given/surname/" (form).

For NAME we might enter:

• Ludvig van Beethoven (Beethoven, Ludvig van) as Ludvig van /Beethoven/
• Ursula von de Leyen (von de Leyen, Ursula) as Ursula /von de Leyen/
• Jacqueline du Pré (du Pré, Jacqueline) as Jacqueline /du Pré/
• Simone de Beauvoir (Beauvoir, Simone de) as Simone de /Beauvoir/
• Ulrich Le Pen (Le Pen, Ulrich) as Ulrich /Le Pen/
• Vincent Van Gogh ( Van Gogh, Vincent) as Vincent /Van Gogh/
• Ronan O'Gara (O'Gara, Ronan) as Ronan /O'Gara/

The "substance" is that the slashed parts of the surnames (in blue above) are the primary sort even though the "form" of the name is that "van", "de" are not part of the "Given name".

This situation is further complicated if we try to bring the GEDCOM GIVN, SPFX and SURN fields into play. I am trying to formulate a wish list item to do this. The interface element is relatively easy to specify; the background processing requirements could get very complex - particularly if we try to respect substance over form.

What then happens when using GIVN, SPFX and SURN in preference to NAME and do we need to maintain alignment? FH works off NAME so ideally we need to represent GIVN, SPFX and SURN as NAME - by some form of concatenation and correct positioning of slashes in order to ensure that things like the Records Window "work".

Trying to recognise "substance" over "form" might we have [GIVN SPFX SURN]:

• Ludvig van Beethoven (Beethoven, Ludvig van) as Ludvig van Beethoven
• Ursula von de Leyen (von de Leyen, Ursula) as Ursula von de Leyen
• Jacqueline du Pré (du Pré, Jacqueline) as Jacqueline du Pré
• Simone de Beauvoir (Beauvoir, Simone de) as Simone de Beauvoir
• Ulrich Le Pen (Le Pen, Ulrich) as Ulrich Le Pen
• Vincent Van Gogh ( Van Gogh, Vincent) as Vincent Van Gogh
• Ronan O'Gara (O'Gara, Ronan) as Ronan O'Gara

The "substance" is followed in that the SURN contains the primary sort elements, even though in some cases it includes the surname prefix - breaking the form rules.

Aligning NAME to match input GIVN, SPFX and SURN for the above examples is relatively easy:

• NAME = GIVN + SPFX + "/" + SURN + "/" (with appropriate spacing)

Note that the SPFX is not incorporated into the slashed surname (which "form" would suggest), because in the examples above the substance is that the surname prefix (in SPFX) is not part of the Primary Sort.

Trying to programmatically (say as a result of a wish list item) align GIVN SPFX and SURN with an input "Given/Surname/" NAME construction is more difficult because we have no way to recognise when there is a "non sorting" surname prefix outside the slashes - that is a consequence of allowing substance to trump form!

There are rules; some say that in French it matters if the prefix is an article (e.g. Le, L') in which case it is part of the sort, or a preposition (e.g. de, du) in which case it is not normally part of the sort); some say in Afrikaans the prefix is normally always part of the sort. it's the "normally" that comes back to bite us. The more complex the rules the less likely we are to have "abnormal" surnames which would require manual edits.

The temptation in writing a wish list item is to say:

• here are the requirements for an interface (discussed elsewhere although my thinking has evolved a bit since then.)
• here are the requirements to convert input GIVN, SPFX and SURN fields into a workable NAME field (following substance - primary sort/secondary sort - rather than strict form), and
• converting the other way does not really matter and can be done by manual edit, or by a coarse conversion (e.g. all uncapitalised short words immediately before the opening slash are "surname prefix") followed by manual review

Is that last line acceptable or is it dodging the issue? Is it acceptable that we should give primacy to substance over form - does that impact importing and exporting?

[tatewise]

David, unless I missed it, this posting does not seem to explain any benefit of using the GIVN, SPFX and SURN fields.
The name sort precedence is still SURN first and then GIVN & SPFX after migrating them into the NAME field.

BTW: A recent thread has explained that Surname first sorting is not always the case, particularly in some Result Sets.

[davidf]

... unless I missed it, this posting does not seem to explain any benefit of using the GIVN, SPFX and SURN fields.

The long thread Surname prefix (SPFX) -- more generally, handling structured names. (20719) kicked around the need for the GIVN, SPFX and SURN fields to meet some of the needs of non Anglo-centric users.

But that discussion was very much premised on following strict "form", that is the surname prefix always went into the SPFX field whether or not it formed part of the primary sort.

In this thread I am pondering whether breaking away from strict form and thinking about names in terms of the substance - primary/secondary sort - changes requirements. It might partly undermine the argument that non Anglo-centric users need easy access to the GIVN, SPFX and SURN fields. It might mean some say "let's have the GIVN, SPFX and SURN fields for strict "form" requirements, but consciously use NAME for "substance" requirements". Being pretty firmly in the Anglo-centric world I can ask that question but there are others more able to answer it.

(Applies also to discussion of Scandinavian name forms - "Given, Patronymic, Toponymic"; instead of asking ("form") which bit is "the surname" and goes "between the slashes", ask (substance) which bit is "my primary sort" and put that "between the slashes".)

The name sort precedence is still SURN first and then GIVN & SPFX after migrating them into the NAME field.

Well that surely depends on how you migrate them. Recent threads we have assumed that it is SPFX and SURN "between the slashes" - because strict "form" says the Surname Prefix is not a Given Name, but part of the "surname" (Le Pen, von de Leyen, van Beethoven?). "Prefix" is ambiguous and inconsistent!

BTW: A recent thread has explained that Surname first sorting is not always the case, particularly in some Result Sets.

But we have established that Surname first sorting is the historic default expectation of FH and that if we are to try to bring GIVN, SPFX and SURN into easy availability, maintaining NAME primacy and the associated primary sort on the "between the slashes" does avoid any change creating deep coding issues elsewhere.

[AdrianBruce]

Helicopter view... Do we need to step back and say, "Stop trying to come up with one sort order"? (Or whatever else we are considering apart from sort orders...)

Do we need to come up with a Flemish format for the name and a French format for the same name? And maybe this needs a separate item (or items) for the indexed version of the name to cope with the librarians who index von, van, de, etc, differently for the same name, depending on who their employer is?

(Have we talked about Hungarian names where the same person may have both the "proper" Hungarian order of family-name then given-name, and the "I can't be bothered explaining" order of given-name family-name....?)

[davidf]

Helicopter view... Do we need to step back and say, "Stop trying to come up with one sort order"? (Or whatever else we are considering apart from sort orders...)

Do we need to come up with a Flemish format for the name and a French format for the same name? And maybe this needs a separate item (or items) for the indexed version of the name to cope with the librarians who index von, van, de, etc, differently for the same name, depending on who their employer is?

It may be that that is the direction GEDCOM eventually goes. But I am trying to get my mind around how we might use the current FH versions 6 & 7 and what we might wish to see in version 8 or possibly 9.

Multiple versions of Names of People/Places

The issue of multiple formats/versions for the same name (be it the name of an individual or a place) has been kicking around for some time:

• Kyung Wha Chung (Chung Kyung Wha) originally Latinised and put in "westernised order" KWC, to meet US marketing expectations?)
• Egon Ronay (Rónay Egon) - again "Anglicised"
• Sir Georg Solti (Solti György) - again "Anglicised" in both spelling and word order
• Arnold Paltrow (né Paltrowitz) - again "Anglicised"
• Rónán Ó Gadhra > Ronan O'Gara - Anglicised for us poor "Anglos" who cannot pronounce/type/remember the Irish form - the quiet toleration of our incompetence and insensitivity is often not acknowledged.
• Peking > Bejing (a better/local transliteration into "Western Text" of Mandarin 北京)
• Calcutta > Kolkata (a better/local transliteration into "Western Text" of Bengali কলকাতা)
• Cologne > Köln (a local name triumphing over an Anglicised form?)
• Biel / Bienne (options in a bi-lingual community - in one hotel it depended if morning or evening staff were on duty!)
• Londonderry / Derry (politics)

GEDCOM and variations of individual names

Many of these individual name variations can be "handled" in GEDCOM 5.5.1 by NAME_TYPE (e.g. "Name Assumed on Immigration" - Paltrow from Paltrowitz?) or ROMANIZED_TYPE (e.g. Korean "정", also often "spelled" Chung, Jung, Joung or Jong. - The Revised Romanization transcription Jeong being the most frequently used). Likewise PHONETIC_TYPE.

But "handling" seems to be limited to different versions being held under the "Name Structure":

Code: Select all

PERSONAL_NAME_STRUCTURE:=
n NAME <NAME_PERSONAL> {1:1} p.54
+1 TYPE <NAME_TYPE> {0:1} p.56
+1 <<PERSONAL_NAME_PIECES>> {0:1} p.37
+1 FONE <NAME_PHONETIC_VARIATION> {0:M} p.55
+2 TYPE <PHONETIC_TYPE> {1:1} p.57
+2 <<PERSONAL_NAME_PIECES>> {0:1} p.37
+1 ROMN <NAME_ROMANIZED_VARIATION> {0:M} p.56
+2 TYPE <ROMANIZED_TYPE> {1:1} p.61
+2 <<PERSONAL_NAME_PIECES>> {0:1} p.37

Something still has to go in the Primary "NAME" and that is what FH uses as the key name, and I am trying to get my mind around what goes into that NAME field ( "A/B/C") where "B" is NAME:SURNAME and is key to so much functionality within FH - yet many naming systems do not comfortably fit this structure.

Name Order

(Have we talked about Hungarian names where the same person may have both the "proper" Hungarian order of family-name then given-name, and the "I can't be bothered explaining" order of given-name family-name....?)

FH can already handle "/Rónay/ Egon" if that is how we wish to refer to him; the labelling of "family name" and "given names" is handled. How we put it in is a user choice - we put in what we want to put out - which usually depends on our audience. Unfortunately, even with language packs in V7, I don't think it is possible to "switch name order" if we are outputting for a Hungarian or Chinese or Korean etc. audience. The recommendation is to "Enter all names, places and addresses in the project language" and when outputting "Names of people, ... in any context ... are not translated". (which presumably means they will not switch word order).

[Likewise "/Mao/ Zedong" or "/Mao/ Tse-tung" (Official Hanyu Pinyin and Wade-Giles romanisations). We can hold multiple romanisations, but we choose which romanised verison we put into the NAME field for FH to process. (We probably would not put "毛泽东" in the Name field because that is "too hard" for western users to handle.)]

Non standard name structures

So with names, "you put in what you want to get out" - which means that (outside heavy customisation of sentences, diagram and reports) - what is put in the NAME field is key to both output and processing even if the name in question does not follow any family/given format. Surname prefixes are possibly just the most notable "European" example of this issue - but we have also got use of patronymics (with or without toponymics) and mixed patronymics/matronymics (in some Iberian names), etc..

Yet I get the impression that losing sight of name parts - surname prefixes, patronymics, farm_names, etc. does represent a loss of potential functionality.

We don't want to output a sentence like "van Beethoven lived in Bonn", but "Beethoven lived in Bonn" I get the impression that Germans and Austrians would want (using German as the output language) to say something approximating to "Beethoven lebten in Bonn". In this usage the "van" does not form part of the "surname". I think most usages also want Beethoven to sort under "B" rather than under "v".

(Ludvig's ancestors - per the inevitable Wikipedia - came from Mechelen in the Austrian Duchy of Brabant (in what is now the Flemish region of Belgium) - so it is "van" not "von" - previously I have been inconsistent)

GEDCOM and GIVN SPFX and SURN and "Substance over Form"

Within GEDCOM we can already recognise surname prefixes and in the first post of this topic I pondered about only putting the prefix in the SPFX field if it was a non-indexing prefix, and then when trying to align GIVN, SPFX and SURN with NAME, putting the SPFX field contents immediately before the first "surname slash" - which is a potential change from earlier discussion brought about by considering the substance of what the name parts do in the context of an individual name rather than the strict form of what the name parts "are". (Contrast this with "von der Leyen", where I am told that is the surname so the prefix is part of the surname for indexing, display and sentences and gets put between the slashes. So we have "Ludvig van /Beethoven/" but "Ursula /von de Leyen/")

As I think Mike was then hinting, if we don't get hung up on the "strict" form of prefixed surnames can we allow the substance of how the name is used to determine what goes "between the slashes" and then do we even need to worry about GIVN, SPFX and SURN? And by extension does that apply to other name forms? Do we then enter "Given /Patronymic/ Toponymic" if the user wants the primary sort to be on the Patronymic but "Given Patronymic /Toponymic/" if they want the primary sort to be on the Toponymic - and therefore not have to worry about "which bit is 'the surname'"?

That may meet a lot of user requirements without software enhancements or plugins. But it is a usage that is at variance with what GEDCOM in its Anglo-centric way was anticipating:

The name value is formed in the manner the name is normally spoken, with the given name and family name (surname) separated by slashes (/). (See <NAME_PERSONAL>, page 54.) Based on the dynamic nature or unknown compositions of naming conventions, it is difficult to provide more detailed name piece structure to handle every case. The NPFX, GIVN, NICK, SPFX, SURN, and NSFX tags are provided optionally for systems that cannot operate effectively with less structured information. For current future compatibility, all systems must construct their names based on the <NAME_PERSONAL> structure. Those using the optional name pieces should assume that few systems will process them, and most will not provide the name pieces.

Which is essentially saying that all names must follow the format "aaa/bbb/ccc", where any element is optional (provided at least one is included) and in FH the default sort happens on bbb,aaa,ccc (NAME:SURNAME_FIRST in terms of FH Name Formats)

Wider View

Helicopter view... Do we need to step back and say, "Stop trying to come up with one sort order"? (Or whatever else we are considering apart from sort orders...)

I think with the current coding of FH (and probably many other "GEDCOM Processors") that is exactly what we have - one (default) sort order. If we want to do anything else we have to be able to identify the "name pieces" by which we wish to sort and currently we are limited to those provided by the FH name formats (NAME:SURNAME, NAME:GIVEN_ALL etc.) and the GEDCOM pieces accessed through the All Tab (GIVN, SPFX and SURN).

...
Do we need to come up with a Flemish format for the name and a French format for the same name? And maybe this needs a separate item (or items) for the indexed version of the name to cope with the librarians who index von, van, de, etc, differently for the same name, depending on who their employer is?

I know what you mean (and weren't you the person who questioned me labelling a style "Dutch Style" in answering an original OP about Dutch surnames?)

The Search for a "Parsing Rule"

I started pondering this thinking it was relatively easy - all you had to do was "discern the rule"! When Mike was trying to develop a plug-in for bulk loading of GIVN SPFX and SURN from NAME it quickly became obvious that finding and coding "the rule" was far from easy.

I have been reading (parts of):

• The Indexer: Centrepieces - "The International Journal of Indexing" - First published by the Society of Indexers (UK) in 1958 on a twice-yearly basis, it moved in 2008 to a quarterly publication. From 2019 the journal is published by Liverpool University Press "on behalf of indexing societies worldwide".
• Names of Persons published by by the International Federation of Library Associations and Institutions

I give as one example from "Centrepieces":

Centrepieces on Indexing names of Dutch/German origin

Screenshot from 2022-11-27 11-51-14.png (52.22 KiB) Viewed 782 times

I am thinking that trying to enter a name in the NAME field (in "strict form") as "Givenames /Surname/" and then having a routine to sort out and load the SPFX and SURN fields such that lists sort properly is a fool's errand even if you have a series of icons to load these fields according to [Dutch|Flemish|Afrikaans|German] rules.

I think it is easier (if initially non-intuitive) if you enter names in the NAME field following "substance" as "secondary sort elements /Primary sort elements/", but that only handles "Prefixed Surnames". (The "rule" would be something like "all continuous name elements immediately preceding the first slash which are uncapitalised" are to be loaded into SPFX, slashed elements into SURN and the rest into GIVN". That would give a usage of GIVN, SPFX and SURN that gives primacy to substance over form. But that rule might then choke on a name where the final unslashed ("given") name is something like "d'Aragon" which may be a Toponymic!)

Do Language Variants Help?

Following "Substance over Form" you would in the above examples put the element before the comma "between the slashes" and the rest in the name field before the first slash. But you have to decide on your "project language" to determine which column in the above example you are using as your guide. But having decided that if you then want to output in a different language FH will not reconfigure the slashes (it does not translate names). So if the project language is "Flemish", you enter "Eddy /D'hondt/", but if you use Dutch as your output language it will not sort Eddy as "Eddy d'/Hondt/"; he will be listed between the C's and the E's!

The only way I can see round this is, as Adrian suggests, to hold multiple variants of the NAME with NAME_TYPEs specifiying which "language rule is being followed:

Code: Select all

1 NAME Eddy /D'hondt/
2 TYPE Flemish Format
1 NAME Eddy d'/Hondt/
2 TYPE Dutch Format
1 SEX M

Extract from a GEDCOM showing use of Name Types to hold different language formats
There is an inevitable overhead in maintaining this data and without some form of "macro" to swap the names around (possibly driven by Output Language choice, but defaulting to Project Language choice) the opportunities to get tangled are immense.

Identifying specific name parts for Secondary Analysis

If you enter names as "secondary sort elements /Primary sort elements/" do you then need to identify name parts for any form of analysis, filtering or reporting? Is that a secondary issue - other than the sort of issue in the section above?

The question then would be how to hold those name parts that need to be specifically identified: patronymic, toponymic (farm name) etc. You could, as has previously been suggested, use secondary names (and NAME_TYPE) in addition to the Primary NAME field to hold specific elements. You could thus have in your tree multiple Individuals with a secondary name of "Bruflot" with NAME_TYPE: "farm name". Currently this is done (V7 only) through the All tab. NAME_TYPE is not currently accessible through the Names & Titles Dialog. Adding support for NAME_TYPE to the Names & Titles Dialog would be a minor enhancement (Wish List 520).

It might then be possible to do a query, for instance, of all Individuals where a secondary NAME of NAME_TYPE = "farm name" contains "Bruflot".

Someone like "Leonardo di ser Piero da Vinci" could have a name Structure like:

Code: Select all

0 @I78@ INDI
1 NAME Leonardo di ser Piero /da Vinci/
1 NAME da /Vinci/
2 TYPE Toponymic
1 NAME di ser /Piero/
2 TYPE Patronymic
1 SEX M

Extract from a GEDCOM showing use of Name Types to hold Toponymics and Patronymics

Notes:

• The Primary Name is entered as you want it output; in this instance he sorts under "da Vinci" rather than "Vinci"
• I am not sure about the slashing of those secondary names; it may or may not be a useful way of drawing out the stem of the Toponymics and Patronymics.
• Care needs to be taken to ensure that these sort of secondary names are not accidentally promoted to the Primary Name
• It would also be useful for the user to be able to standardise these NAME_TYPEs - with possibly some (e.g. Patronymic, Toponymic, etc.) supplied "out of the box".
• With Russian (Slavic?) Patronymics which have gendered endings (e.g. Pyotr Ilyich Tchaikovsky, and his sister, Aleksandra IIinichna Tchaikovskaya, you could enter "Ilya" - the actual name of the father, Ilya Petrovich Tchaikovsky - as the Patronymic, so that you could get all the siblings to sort/report together. Having the patronymic identified as a secondary name allows you to avoid issues arsing from gendering because the actual patronymic is incorporated in the Primary Name.
• With Welsh Patronymics which tend to have gendered prefixes, using "contains" in a filter enables filtering of siblings together.

Where multiple names need to be input a separate tab might be a more streamlined way of entering them than the current Names & Titles dialog.

Multiple Names and Titles Dialog

Screenshot from 2022-11-28 08-14-56.png (31.28 KiB) Viewed 743 times

Note this is a V6 dialog so I have a note field where the NAME_TYPE field would logically go.

Does that "long spiel" talk many of the Non-standard Name entry/storage issues out of existence - given implementation an easier access to the NAME_TYPE - possibly with some protection of the Primary Name and standardisation control over NAME_TYPEs? Or are non-Anglo-sphere users (and potential users) likely to pull their hair out because once again their requirements have not been understood?