* Migration from PAF to FH5

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Migration from PAF to FH5

Post by Taxin » 01 Mar 2014 05:29

Having just converted from PAF5 to FH5 (and used the UDF plugin to identify unknown tags) I would like to ask the forum if anyone has developed a plugin that completes the migration, ie. extracts paragraphs from the single PAF5 NOTE statement, identifies the event, then saves the paragraph in the appropriate event's Text from Source or Note field.

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Migration from PAF to FH5

Post by Jane » 01 Mar 2014 10:59

I have not seen one, what do the notes look like?
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 02 Mar 2014 03:25

Good morning Jane

PAF notes are held against an individual and each note contains a number of tags. This is just one sample from my gedcom (I try to structure my source notes).

1 NOTE BIRTH: SCO Birth 18611 041/00 37
2 CONT Helen Macleod
2 CONT Born: 7th March 1861 2h am at Bardnaclavan
2 CONT Father: John McLeod, Quarry labourer
2 CONT Mother: Elizabeth Macleod ms Mowat, married 1st April 1859 at Thurso
2 CONT Informant: John McLeod (father)
2 CONT Reg.on: 7th March 1861 at Thurso
2 CONT
2 CONT DEATH: SCO Death: 1899 041/00 83
2 CONT Helen McLeod
2 CONT Died: 13th December 1899 5h 30m pm at Achscrabster, Thurso
2 CONT Age: 38 yrs
2 CONT Father: John McLeod, Labourer
2 CONT Mother: Elizabeth McLeod ms Mowat
2 CONT Cause: Chronic nephritis
2 CONT Informant: John G McLeod (brother) of 10 Hillside Street, Edinburgh
2 CONT Reg.on: 18th December 1899 at Thurso

The default tags used by PAF are BIRTH:, CHRISTENING:, MARRIAGE:, DEATH:, CENSUS: and BIOGRAPHY:. Other user-defined tags can exist and will always occur at the start of a paragraph.

John

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 02 Mar 2014 03:29

I have started to put some thoughts to paper on how (I suspect) a suitable plugin should work. Your comments and guidance would be welcome.

MoveToFirstRecod INDI
do while not EOF
if INDI has a NOTE then
save Note in table (one pair per paragraph)
for each table pair
if pair(i,v) begins with "BIRTH:" then
if not exists (INDI.BIRT.SOUR) then
create INDI.BIRT.SOUR
end
append pair(i,v) to INDI.BIRT.SOUR
table.remove(pair(i,v))
end
-- CHRISTENING, MARRIAGE and DEATH are similar
if pair(i,v) begins with "CENSUS:" then
if not exists (INDI.CENSUS) then
create INDI.CENSUS
end
append pair(i,v) to INDI.CENSUS
table.remove(pair(i,v))
end
if pair(i,v) begins with "BIOGRAPHY:" then
-- do nothing
end
end
save table back to INDI.NOTE
end
MoveToNext INDI
end

and now I'm reading up on the finer points of programming in Lua.

John

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Migration from PAF to FH5

Post by Jane » 02 Mar 2014 10:05

A couple of quick notes, when you access the note, it will appear as a single string, so you will need to split it to a table to easily work on it, so each paragraph ends up in it's own string then you can check for the label on the front and extract and move the information as needed. Getting a handle on Lua patterns will put you in good stead to do this.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 03 Mar 2014 00:05

I have a first prototype Plugin for you to consider. It comprises two main parts:

(1) A loop through Individuals that breaks any Note into lines, and searches for a PAF Label.
All subsequent lines make the Note From Text to be associated with a Fact GEDCOM Tag.
Any leading lines, or BIOGRAPHY lines, are retained as Note text.

(2) A function that ensures a Fact GEDCOM Tag exists with a citation of a Source Record.
If the Fact or Source need to be created then all fields will be empty.
It saves the Note From Text in the Source Record providing the field does not already exist.

Note the use of a table of Fact Tags versus PAF Labels that drives the processing.
This allows the same code to be used for every Tag and Label rather than separate code sections.

There may well need to be changes if you want to extract some parts of the PAF Notes to fill in any blank fields.
Also it currently only works on the 1st instance of each Fact and Citation.
So multiple Marriage or Census events or multiple Citations may need further consideration.

Is this format of PAF Notes in general use, or is it your personal format?

Code: Select all

--[[
@Title:			Move PAF Notes to Facts
@Author:			John Mowatt
@Version:			1.0
@LastUpdated:	Mar 2014
@Description:	Move PAF Labelled Notes to Fact associated Text From Source
]]

local function SaveTextFromSource(ptrRec,strTag,strText)		-- Save the Text From Source field
	local ptrFact, ptrSour, ptrText, isOK
	if strTag and strTag ~= "X" then
		if strTag == "MARR" then
			local ptrFam = fhGetItemPtr(ptrRec,"~.FAMS>")		-- Switch to Family record for Marriage event
			if ptrFam:IsNull() then
				error("\n\nNo Family Record exists for "..fhGetDisplayText(ptrRec))
			end
			ptrRec = ptrFam:Clone()
		end
		ptrFact = fhGetItemPtr(ptrRec,"~."..strTag)				-- Ensure the Fact entry exists
		if ptrFact:IsNull() then
			ptrFact = fhCreateItem(strTag,ptrRec,true)
		end
		ptrSour = fhGetItemPtr(ptrFact,"~.SOUR>")					-- Ensure the Source record exists
		if ptrSour:IsNull() then
			ptrFact = fhCreateItem("SOUR",ptrFact,true)			-- Create the Citation entry
			ptrSour = fhCreateItem("SOUR")
			isOK = fhSetValueAsLink(ptrFact,ptrSour)				-- Link the Source Record
		end
		ptrText = fhGetItemPtr(ptrSour,"~.TEXT")					-- Ensure the Text From Source exists but empty
		if ptrText:IsNull() then
			ptrText = fhCreateItem("TEXT",ptrSour,true)
		else
			if fhGetValueAsText(ptrText) ~= "" then
				error("\n\nText From Source already exists for "..fhGetDisplayText(ptrSour))
			end
		end
		isOK = fhSetValueAsText(ptrText,strText)					-- Save its Text
	end
end -- local function SaveTextFromSource

-- Table of Gedcom Fact Tags versus PAF Fact Labels
local tblFact = { BIRT="^BIRTH: ", CHR="^CHRISTENING: ", MARR="^MARRIAGE: ", DEAT="^DEATH: ", CENS="^CENSUS: ", X="^BIOGRAPHY: " }

local ptrIndi = fhNewItemPtr()												-- Pointer to Individual Records
ptrIndi:MoveToFirstRecord("INDI")
while ptrIndi:IsNotNull() do													-- Loop through all Individual Records
	local ptrNote = fhGetItemPtr(ptrIndi,"~.NOTE2")
	if ptrNote:IsNotNull() then												-- Individual has a Note field
		local strText = ""														-- Text From Source field text
		local strNote = ""														-- Note text to be retained
		local strTag = nil														-- Gedcom Tag for current Fact
		for strLine in fhGetValueAsText(ptrNote):gmatch(".-\n") do	-- Break the Note text into lines including blank lines
			for strFact, strLabel in pairs(tblFact) do
				if strLine:match(strLabel) then								-- Line starts with a PAF Fact label
					SaveTextFromSource(ptrIndi,strTag,strText)			-- Save any Text From Source found previously
					strLine = strLine:gsub(strLabel,"")						-- Strip PAF Fact label from Note line
					strText = ""													-- Clear the Text From Source
					strTag = strFact											-- Save current Fact Gedcom Tag
					break															-- Escape from search loop
				end
			end
			if strTag and strTag ~= "X" then
				strText = strText..strLine									-- Append the Line to Text From Source
			else
				strNote = strNote..strLine									-- Append the Line to retained Note
			end
		end
		SaveTextFromSource(ptrIndi,strTag,strText)						-- Save any Text From Source found previously
		if strTag then
			if strNote == "" then
				fhDeleteItem(ptrNote)											-- Delete the Note if empty
			else
				fhSetValueAsText(ptrNote,strNote)							-- Save any Note retained
			end
		end
	end
	ptrIndi:MoveNext()															-- Move to the next individual
end
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
richardkendell
Gold
Posts: 18
Joined: 03 Mar 2014 08:13
Family Historian: V6.2

Re: Migration from PAF to FH5

Post by richardkendell » 03 Mar 2014 08:48

I think this is exactly the problem I have. Having entered all event specific information In PAF Notes (creating additional categories to meet my needs) I find I cannot export sensibly to other programs. By way of an example, in the case of Avery Dunning I have Notes categories for BIOGRAPHY, BURIAL, CHRISTENING, DEATH, OWN WILL, 1841 CENSUS, OCCUPATION, MARRIAGE. This then emerges on GEDCOM as:
1 NAME Avery /Dunning/
2 SURN Dunning
2 GIVN Avery
1 SEX M
1 CHR
2 DATE 4 May 1788
2 PLAC Ansty
1 DEAT
2 DATE 16 Jul 1848
2 PLAC Bulkington
1 BURI
2 DATE 20 Jul 1848
2 PLAC Bulkington
1 _UID D01DC8B89B13D711B93DA5F49A83FC3CE749
1 EVEN
2 TYPE MARR
1 NOTE
2 CONT
2 CONT
2 CONT BIOGRAPHY: WARKS POLL BOOK 1833 ELECTION IN BULKINGTON EVERY DUNNING OF RUITON VOTED FOR DE
2 CONC MPSTER HEMING ESQ (1568 VOTES) OTHER INF:
2 CONT
2 CONT
2 CONT BIOGRAPHY: Avery Dunning 1836 Bulkington Jurors Lists 1836 WEAVER FREEHOLD
2 CONT
2 CONT BIOGRAPHY: Avery Dunning 1842 Bulkington Jurors Lists 1842 WEAVER FREEHOLD
2 CONT
2 CONT BIOGRAPHY: Avery Dunning 1847 Bulkington Jurors Lists 1847 WEAVER FREEHOLD
2 CONT
2 CONT BURIAL: BULKINGTON 1813-1900 20/7/1848 AVERY DUNNING 60
2 CONT BULKINGTON MONUMENTAL INSCRIPTIONS BIRMINGHAM LIBRARY IN AFFECTIONATE MEMORY OF AVERY DUNNIN
2 CONC G DIED 16/7/1848 AGED 63 ALSO OF HANNAH WIFE OF THE ABOVE DIED 10/7/1875 AGED 85 THY WILL B
2 CONC E DONE
2 CONT
2 CONT
2 CONT CHRISTENING: ANSTY PARISH RECORDS 4/5/1788 AVERY WILLIAM DUNNING ELIZABETH
2 CONT
2 CONT
2 CONT DEATH: MEMORIAL INSCRIPTIONS FOR BULKINGTON ST JAMES (BEDWORTH AND WARKS GEN SOC BOOKLET AT C
2 CONC OVENTRY REC OFF) AVERY DUNNING DIED 16/7/1848 63 HUSBAND OF HANNAH
2 CONT
2 CONT
2 CONT OWNWILL: LICHFIELD WILL (HAVE COPY) DIED 16 JULY 1848 PROVED 6 OCT 1848 AVERY DUNNING OF RIET
2 CONC ON IN PARISH OF BULKINGTON I GIVE ALL MESSUAGES TENEMENTS WEAVER SHOP AND BUILDINGS GARDENS N
2 CONC OW IN OCCUPATION OF MYSELF WILLIAM JONES JOHN LOVEITT THOMAS MOSS THOMAS BRANSTON TO WIFE HAN
2 CONC NAH DUNNING THEN DIVIDED AMONG CHILDREN. TO SON JOHN DUNNING £20 (NO SHARE IN MESSUAGE ETC)
2 CONC . TO HANNAH ELSON D OF SARAH ELSON AND GRANDDAUGHTER OF WILLIAM AND HANNAH £10. SON THOMAS DU
2 CONC NNING OF RIETON RIBBON WEAVER AND WILLIAM SLINGSBY SCHOOLMASTER TRUSTEES
2 CONT
2 CONT
2 CONT 1841CENSUS: 1841 CENSUS WARKS BULKINGTON RYTON AVERY DUNNING 50 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON HANNAH DUNNING 50 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON THOMAS DUNNING 22 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON DAVID DUNNING 15 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON HANNAH DUNNING 16 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON AVERY DUNNING 12 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON ANN DUNNING 10 Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON HENRY DUNNING 8 Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON EMMA DUNNING 6 Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON JOHN BELCHER 24 Weaver Y
2 CONT 1841 CENSUS WARKS BULKINGTON RYTON SARAH BELCHER 24 Weaver Y
2 CONT
2 CONT
2 CONT OCCUPATION: WEAVER
2 CONT
2 CONT MARRIAGE: WARKS BULKINGTON PARISH REGISTERS 28 12 1812 HAVERY DUNNAL B BULKINGTON HANN
2 CONC AH WRIGHT S BULKINGTON MARY HURST GEO CUTHBERT


What I need is a method of assigning each existing Notes category (which are clearly evident by the "1841 CENSUS: OWNWILL:" etc Tags) to something (Event?) in Family Historian.
I apologise for adding my contribution if the thread issue and mine are different, but if they are the same I suspect there are quite a number of PAF users facing the same problem.

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 03 Mar 2014 10:10

Good morning all

First, thank you to Mike for his modification of my initial plugin script. I shall have a detailed look at the script once I finish this message.

Mike/Jane: While I cannot vouch for every installation of PAF5, those that I have seen all seem to follow the same pattern, ie. a single 1 NOTE statement containing a number of paragraphs with each paragraph beginning with "<tag>:<space>" followed by some text. Lines (in the example below) which begin with CONT and CONC are a continuation of the NOTE statement. CONT is used when the number of characters on the previous line is reasonably short. CONC is used when the number of characters exceeds a limit.

1 NOTE BIRTH: Birth also recorded as 8 November 1889
2 CONT
2 CONT MARRIAGE: NSW Marriage 14413/1914
2 CONT They were married in 1914 at Enmore (probably because the local churches were booked). Percy was a shearer when he married Alma. She went with him on trip to Winton (see photo). Percy was known as "Lolly Mears" because he was tall and thin durin
2 CONC g his shearing days.

When the above is viewed within PAF5 it appears as:
BIRTH: Birth also recorded as 8 November 1889

MARRIAGE: NSW Marriage 14413/1914
They were married in 1914 at Enmore (probably because the local churches were booked). Percy was a shearer when he married Alma. She went with him on trip to Winton (see photo). Percy was known as "Lolly Mears" because he was tall and thin during his shearing days.

The standard tags were mentioned in my previous post but it is possible for a user to create their own tags, although I did not do that. I should also add that the NOTE statement stands in isolation and is not linked to any of the events as happens in FH5.

Hoping this helps.

John

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 03 Mar 2014 11:06

Mike,

I (and presumably others) will often have more than one PAF tag per event, eg. multiple marriages and multiple census entries, and even birth/death details followed by an RCE (Register of Corrected Entries) entry so the plugin should be able to cater for them.

Would I be on the right path in thinking that in the main routine I should comment out the line containing 'break' and in the function I should replace the 'Text from Source already exists' message with a line or two that concatenates the already existing text and the new text.

I have taken a copy of the code you supplied and will pencil in some changes while I'm at work tomorrow.

Once again, thank you for your assistance and support.

John

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 03 Mar 2014 14:09

OK guys, that is why I called my Plugin a prototype, because the journey has only just begun.
It now seems that most PAF to FH migrations would benefit from this Plugin conversion.

John, the break command has no logical effect, but simply curtails the loop once a matching PAF Label has been found at the start of the line, because no subsequent PAF Label could possibly match, so just wastes time looking.
Yes, if Text From Source exists, then add the PAF Note text to it.

It is important to define as precisely as possible exactly what the PAF Note formats can be, and how to handle the Facts associated with them.
e.g.
John mentions multiple MARRIAGE and CENSUS entries.
I presume multiple CENSUS entries are as Richard illustrates preceded by a year: 1841 CENSUS: or 1911 CENSUS:, etc.
But is that the only format, and how are other multiple entries formatted?

Is it to be assumed that a Fact will exist for the Individual to match every PAF Note Label?
Or must missing Facts be created, as the prototype Plugin does?
Similarly, how should Source citations be handled?
Should the PAF Note details always be added to the 1st Source citation?
As John says, these PAF Note details should be added to the Text From Source, but before or after the existing text?

It is quite straightforward for PAF Note Labels that are related to Standard Fact Tags in GEDCOM.
e.g. BIRTH: BIRT , 1841CENSUS: CENS[year=1841] , OWNWILL: WILL , OCCUPATION: OCCU
Have you a comprehensive list?
But what is the relationship between non-standard user defined Labels and their GEDCOM Fact Tags?
e.g. BIOGRAPHY: etc...
This is where a Plugin dialogue window may be needed to define all these custom Labels and Tags.

John also mentioned RCE (Register of Corrected Entries), but gave no details of its format, or what to do with it.

And so the Plugin grows more complex...

I propose to start a plugins:wip:index|> Work In Progress Plugins page for this PAF to FH Plugin.

Are there any other PAF to FH conversions that could be automated or useful reports created to help the migration?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
richardkendell
Gold
Posts: 18
Joined: 03 Mar 2014 08:13
Family Historian: V6.2

Re: Migration from PAF to FH5

Post by richardkendell » 03 Mar 2014 14:50

Mike
For what it's worth here are the Notes categories I use:
BIOGRAPHY:
BIRTH:
CHRISTENING:
1684CENSUS:
HEARTHTAX:
1841CENSUS:
1851CENSUS:
1861CENSUS:
1871CENSUS:
1881CENSUS:
1891CENSUS:
1901CENSUS:
1911CENSUS:
MARRIAGE:
DIVORCE:
OCCUPATION:
APPRENTICESHIP :
MILITARYSERVICE:
VOTINGRECORD :
NEWSPAPER:
OTHERWILL :
DEATH:
BURIAL:
OWNWILL:
ADDRESS :
PROTESTATIONRETURN

I'd like to make a couple of points, firstly PAF only allows a single word for the Tag so 2 words eg Protestation Return will get joined together. Secondly you can create any number of Notes so the possibility of variations are pretty much endless. I tend to use BIOGRAPHY for general snippets that don't fit anywhere else.
On my PAF all of these note fields are available to every Personal Record (ie they all appear as a list when I go to the Select Notes page) but are (obviously) only populated when I have appropriate data and when there is data typed in PAF adds a " * " to show that is the case.
I am getting to stage where the more I read about TAGS, Labels and Notes the less I understand about what the various programs do and how, so if I have muddled up the concepts treat me gently!
Thanks for your efforts.

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 03 Mar 2014 15:47

Yes John, it does take a while to get your head round GEDCOM Tag structures, and Lua library functions, plus FH API functions.
To complicate matters further some FH Tag names are slightly different from the GEDCOM Tag names.
e.g. INDI.NOTE2 is GEDCOM INDI.NOTE
What may help is to understand that the GEDCOM data is composed of 9 different Record types.
e.g. Individual (INDI), Family (FAM), Source (SOUR), Multimedia (OBJE), etc.
Where one Record links to another the > symbol follows the link field Tag.
Otherwise nested fields within a Record use . separators.

So INDI.FAMS>MARR links an INDI Record to a FAMS Record that has a MARR event.
Or INDI.BIRT.SOUR>TEXT links an INDI Record BIRT Event to a SOUR Record TEXT From Source field.

Thank you for your list of PAF Note Labels, but just illustrates that the Plugin needs to cater for such lists from many users.
This would be done via a dialogue window that accepts both the PAF Note Label and GEDCOM Fact Tag in matched pairs.
That then poses the question of what are your matched pairs?

You also need to answer all the other points I raised before we move on to Plugin V1.1.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
richardkendell
Gold
Posts: 18
Joined: 03 Mar 2014 08:13
Family Historian: V6.2

Re: Migration from PAF to FH5

Post by richardkendell » 03 Mar 2014 16:10

Mike
I have just reread your reference to Sources. I suspect like many others I have picked endless bits of data up over the years some better "sourced" than others. By way of example in my various Notes sections on Hearth Tax will be included a range of sources with associated facts
"
CAKE AND COCKHORSE VOL1 P 63 BURTON DASSETT WARKS HEARTH TAX 1663
HENRY HASLEWOOD HEARTH NOT LIABLE

OXFORDSHIRE HEARTH TAX RETURNS 1665 SOG
NEHEMIAH HASLEWOOD IJ (2)


LEICESTER MEDIAEVAL VILLAGE NOTES TONGE LAY SUBSIDY 134/322. 1663/4. HEARTH TAX. TONGE.
JOHN KENDALL 1 HEARTH


LEICS HEARTH TAX
WILLIAM KENDALL BARLESTONE 2 HEARTHS


AUSTREY HEARTH TAX 1670 ROBERT PHD WARKS PRO
GEORGE KENDALL GENTLEMAN 7 HEARTHS"

So I currently in the Notes field include the Source and the relevant data.I readily acknowledge that it is probably not the best way to do it but it is how I started 15 years ago and it met my needs.
I am not sure how complicated this is all looking from your end but I would be more than happy with any facility that in some way associated the list of Notes (see my previous message) with an individual in Family Historian and populated that field with the relevant data from the PAF import. So, for example, George Kendall would have a list of Notes associated with him and the Hearth Tax one would contain
"AUSTREY HEARTH TAX 1670 ROBERT PHD WARKS PRO
GEORGE KENDALL GENTLEMAN 7 HEARTHS"

If I can get the basic data over to Family Historian I assume I can, over time, associate sources myself.

Richard

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 03 Mar 2014 16:22

Please correct me if I am wrong, but those PAF Notes are already attached to the associated Individual.
That is illustrated in both John's and Richard's GEDCOM samples.
So all the Note details already successfully import into FH against each Individual.
The Plugin task is to transfer those details from the Note field to an appropriate Fact (Event/Attribute) Source citation Text From Source field that is associated with the same Individual.
Thus automating the process of creating the Facts and their Sources for you, although they would still no doubt need manual amendments.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
richardkendell
Gold
Posts: 18
Joined: 03 Mar 2014 08:13
Family Historian: V6.2

Re: Migration from PAF to FH5

Post by richardkendell » 03 Mar 2014 20:48

Mike
Yes the notes are associated with an individual but as one large text document. So that document needs splitting down with individual elements listed as a Fact and the text put in the FH5 note field.
So an import of
MILITARYSERVICE: ROYAL HOSPITAL CHELSEA PENSIONER REGISTERS OF SOLDIERS WHO SERVED IN CANADA, 1743-1882 NAME: THOS MERRICK REGIMENT: 31ST REGIMENT OF FOOT RESIDENCE DATE: 10 FEB 1815 TYPE:REGIMENTAL REGISTERS OF PENSIONERS RESIDENCE HINCKLEY DIED 3/9/1959

Would generate a Fact category of MILITARY SERVICE (or an existing FH equivalent) and the rest of the text in the note field. I can't see how you can reasonably separate the Source of "ROYAL HOSPITAL CHELSEA PENSIONER REGISTERS OF SOLDIERS WHO SERVED IN CANADA, 1743-1882" or do anything with dates - I guess they will have to be entered manually.
Richard

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 04 Mar 2014 00:05

Yes, that is the general idea behind the Plugin - to split the one large Individual Note into separate textual parts.
Each textual part is identified by a line starting with an uppercase one word Label terminated by a colon and space.
Each textual part is moved to a matching Fact that could be automatically created if not already existing.

Where Richard and John differ is that Richard expects the textual part to be put in the Fact local Note, whereas John expects it to be put in the Text From Source of a Source Record cited by the Fact.
These alternatives could be Plugin options chosen via a dialogue window.
In both cases, the details of Date and Place, etc, etc, would have to be manually edited afterwards.

When creating what Richard calls "a Fact category" the Plugin needs to know what FH Fact to associate with each Note Label.
This is easy for standard categories such as Birth, Marriage, Death, Census, that would be predefined in the Plugin.
BUT since every PAF user can create their own categories, they must tell the Plugin the FH Fact they want created for each Label.
So taking Richard's example of a Label such as MILITARYSERVICE: he would tell the Plugin to create a Custom Fact called Military Service.
The Plugin needs to know where to position any spaces, and any other differences from the Label.

There may be other ways of associating Labels with Facts once the underlying processes have been agreed.

For instance, the idea of multiple Labels for MARRIAGE: has been mentioned, but the format has not been identified.
Do several different textual parts simply start with MARRIAGE: or are they numbered in some way?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
richardkendell
Gold
Posts: 18
Joined: 03 Mar 2014 08:13
Family Historian: V6.2

Re: Migration from PAF to FH5

Post by richardkendell » 04 Mar 2014 09:59

Mike
Thanks for summing up what is needed so clearly. I think I am going to have to read up what are the distinctions in FH between Label, Fact, Source, Note and Text. To take a simple example of a christening am I right in assuming that typically the Source would be the Parish registers of XXX, the Fact the date of christening, and the Note (and Text) other information in the entry eg parents. The register may refer to the father's occupation so from the same Source details of the occupation would be entered in that Fact category.
As far as marriage is concerned I don't use multiple Labels, in the one section I enter all details of the marriages of that individual so it may contain marriage contracts, Banns, parish register entries etc for as many marriages as a person may have. Multiple labels is not therefore an issue for me.

Richard

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 04 Mar 2014 11:04

Good morning all

Mike, I think you now understand the gist of the problem.

In answer to your last question a PAF note may contain any number of PAF labels. These may be in any order and may occur more than once. There is no inbuilt sequencing - once defined a label may occur may times.

In the PAF environment an event has attributes of date, place, source and the one big notes which we have been discussing. In my case I used date, place and source and these were correctly migrated across to the FH environment.

To keep the plugin simple, and more general, I'd be happy for all BIRTH notes to be appended to the FH birth note attribute, etc. I can run a separate script to move text from the birth note to the birth Text from Source. I haven't developed the rules for that but I don't imagine it will be too complex.

John

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 04 Mar 2014 15:02

Richard ~ Yes, you should become more familiar with FH structures that are based almost entirely on GEDCOM structures.
Label is simply a term I have coined to identify PAF keywords in Notes.
Fact is any Event such as Birth, Marriage, Death & Census, or any Attribute such as Residence, Occupation & Possession.
Each of these have fields such as Date, Age, Place, Address & Note, and can cite a Source.
Source can either identify one document such as a Birth, Marriage, or Death Certificate, or it can identify a class of documents such as the GRO Birth Register or a Parish Church Register.
Each of these have many fields including a Title, Author, and a Text From Source transcript.

If you are both happy, then moving each PAF Labelled Note subsection to the associated Fact ~ Note would be easier than involving Source ~ Text From Source.

I understand that PAF Labels can occur in any order and multiple times in the Individual ~ Note field.

Multiple Label entries need a little more consideration.
Certainly, the Facts such as Birth and Death, that only ever occur once, could have any multiple PAF Labelled Note subsections merged into one Fact ~ Note.
BUT other Facts such as Marriage, Residence, Occupation often arise more than once, and would be better kept separate. This would be easy for the Plugin to implement, and may be preferable in all cases, leaving the user to merge details where preferred.

There are some special cases arising.
9999CENSUS: format Labels would use the 9999 Year to set the Census ~ Date field, and could even enter the Day & Month as well as Year, where this can be deduced.
ADDRESS: format Labels could place the subsection text in the Residence ~ Address field instead of the Residence ~ Note field.

When dealing with unusual custom Labels & Facts not only does the FH Custom Fact name need specifying, but also its type ~ either Event or Attribute.

Could one of you please explain RCE (Register of Corrected Entries)?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
richardkendell
Gold
Posts: 18
Joined: 03 Mar 2014 08:13
Family Historian: V6.2

Re: Migration from PAF to FH5

Post by richardkendell » 04 Mar 2014 17:07

Mike
Thanks. I am afraid I don't know "RCE (Register of Corrected Entries)", it is something I had never come across before.
Richard

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Migration from PAF to FH5

Post by Jane » 04 Mar 2014 17:54

Mike could be
RCE stands for Register of Corrected Entries. Since 1965, it has been known as the Register of Corrections, Etc.

Once an entry in a statutory register had been completed, the original entry could not be altered if an error was later discovered or some other amendment was required as a result of new information,. Instead, each registrar kept a register of corrected entries in which such amendments were written, originally after they had been approved by a sheriff. The original statutory register entry was then marked RCE, Register of Corrected Entries or Reg. Cor. Ent. in the left margin, followed by the volume number, page number and date of the correction.
http://www.scotlandspeople.gov.uk/conte ... .aspx?1314
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 04 Mar 2014 19:58

Good morning all from an overcast Brisbane

Mike, in your post of 15:02 yesterday you said that certain events, such as birth, could only occur once.

In Scotland it seems that it is possible for an event, primarily births, to be registered several times. From what I have seen the child will be registered in the place of the birth, eg Inverness, then it will also be registered in the county of the parent's residence, eg Wick.

May I suggest that you apply a general rule that any event may occur multiple times.

John

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 04 Mar 2014 20:29

Sorry to have mislead you John.
I was just picking up on your comment that you would be "happy for all BIRTH notes to be appended to the FH birth note attribute" that implied just one FH Birth Fact.
When I said "Facts such as Birth and Death, that only ever occur once" I meant the real life events only happen once.
In FH, these and any other Facts can of course have multiple entrie.
I went to say "Facts ... often arise more than once, and would be better kept separate ... and may be preferable in all cases" which would include Birth and Death.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

avatar
Taxin
Gold
Posts: 17
Joined: 22 Feb 2014 02:56
Family Historian: V6.2
Location: Brisbane, Australia

Re: Migration from PAF to FH5

Post by Taxin » 05 Mar 2014 10:41

Morning all,

No problems Mike.

Do you have enough information or (most likely) is there a question or two that I have not answered yet?

I'm looking forward to seeing the plugin when it is unveiled. Would you like a sample PAF gedcom to experiment with?

J.

User avatar
tatewise
Megastar
Posts: 27080
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Migration from PAF to FH5

Post by tatewise » 06 Mar 2014 15:41

I have posted an updated Plugin at plugins:wip:import_from_paf|> Import From PAF.

It would be helpful to have a sample PAF GEDCOM that you could post in the KB page above.
(I am happy to give instructions once you have KB access approved by Jane.)

On looking through earlier postings in this thread one question arises.
In the post from Richard on Mon Mar 03, 2014 8:48 am the following snippet of GEDCOM is shown.
1 EVEN
2 TYPE MARR

which suggests that PAF does not use the conventional GEDCOM 'Family as Spouse' structure for Marriage and Divorce Facts, but uses custom Individual MARR and Individual DIV Facts.
Can you throw any light on this?

Actually this Family Facts feature needs further investigation, because the concept of MARRIAGE: labels is ambiguous if there is more than one 'Family as Spouse' entry for the Individual.
Does the 1st MARRIAGE: label refer to the 1st 'Family as Spouse' entry, or does it refer to the 1st Marriage Fact in any 'Family as Spouse' entry, and similarly for DIVORCE: labels?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry

Post Reply