Convert Any Fact Tag Plugin

Post by **tatewise** » 12 Aug 2011 16:03

I am working on a Convert Any Fact Tag Plugin derived from the Christening-Baptism Conversion Plugin.

It should allow any Individual Fact or Family Fact to be converted or deleted.
It will cope with both Events and Attributes, and Attribute Values.
Popular tag conversions such as Christening-Baptism will have dedicated buttons.
However, any tags may chosen by the user from drop-down lists.

It will let each change be Confirmed or Skipped like the Search and Replace Text Plugin.
It will have sticky settings similar to the Launch Ancestral Sources Data Entry Plugin.

PROBLEM
I have experimented with a draft version with conditional conversions.
After the conversion the replaced tags are no longer in Date order in the GEDCOM.
Tools > Re-order Out-of-Sequence Data has to be used manually to correct the data.
It would be useful if The Family Historian API had access to this capability so the Plugin could correct the data.

ID:5375

YouTube · Post by **Jane** » 12 Aug 2011 19:19

Mike, you can do this yourself in the plugin, by putting the item in the correct place using fhMoveItemAfter once you have replaced the item.

Post by **tatewise** » 13 Aug 2011 17:35

Your fhMoveItemAfter or fhMoveItemBefore suggestion works fine - thank you.

The plugin is well advanced, but the FH API of fhGetDisplayText(ptrFact) and fhGetValueAsText(ptrFact) are inconsistent.
In all examples ptrFact is referring to the Fact tag item.

For any Event:
fhGetDisplayText(ptrFact) shows 'Event Name & Date in Place' e.g. 'Born 1840 in Chelsea'
fhGetValueAsText(ptrFact) shows nothing.

For the Residence Attribute:
fhGetDisplayText(ptrFact) shows 'Resident in Place & Date' e.g. 'Resident in Chelsea from 1847'
fhGetValueAsText(ptrFact) shows nothing.

For any other Attribute:
fhGetDisplayText(ptrFact) and fhGetValueAsText(ptrFact) both show the 'Attribute Value' e.g. 'Solicitor'

Surely the fhGetDisplayText(ptrFact) for an Attribute should show its 'Attribute Name & Value in Place & Date' e.g. 'Occupation: Solicitor in Chelsea 1877'

YouTube · Post by **Jane** » 13 Aug 2011 19:23

Residence is a strange fish, it's defined as an attribute by the GedCom Standard, but looks when recorded more like an event as it does not have a value as other attributes do.

fhGetDisplay Text will always try to give a value for a pointer even if it needs to obtain if from further down the branch (for example a pointer to a link will return the record description, but fhGetValueAsText will not as a link does not have a text value.

I believe Simon is improving the fhGetDisplayText in the way you were asking for, but it's important you understand that fhGetDisplayText is not anything like fhGetValueAsText.

Post by **tatewise** » 13 Aug 2011 21:46

That sounds great.
The fhGetDisplayText(ptrFact) should return a consistent text summary of any Fact.
The fhGetValueAsText(ptrFact) should return a null string '' for Events, and return the Value text field of Attributes, which for Residence will always be a null string ''.

Post by **SimonOrde** » 15 Aug 2011 15:13

I can confirm that fhGetDisplayText will be modified in 5.0.0.3 as Jane says, to give improved summaries for data item - e.g. attributes.

YouTube · Post by **Jane** » 18 Aug 2011 08:56

Mike, when you do the plugin you might want to add some extra code to cope with 'UDF' facts such as the ones which come from Generations.

One 'got-ya' to watch out for is if you have a UDF fact such as

1 BAP
2 DATE 23 MAY 2011
2 SOUR @S3@

FH will 'escape the @S3@ to @@S3@@

So in the child copy you might want to watch out for that and convert the SOUR line to a record link.

Post by **tatewise** » 18 Aug 2011 14:34

The easiest way for my Plugin to recognise and convert UDF is to have a definition for each UDF Fact Tag (such as BAP = Baptism(GFT)).
The Plugin already loads all Fact Tags by reading the Standard, and every Custom Fact Set data file.
One idea I had was a dummy UDF Fact Set data file, but that is probably over complex.
A better idea is to have popular UDF Fact Tags predefined and saved in the sticky data settings file, which the user can easily edit to add extra ones, or the Plugin can be updated.

To get this started, do you have a list, or a link to, a popular set of UDF Fact Tags?

YouTube · Post by **Jane** » 18 Aug 2011 15:11

The two Generations files I have converted for different people had different field names for the same things, Generations allows the user to 'create' them on the fly!

What you could do if you wanted to populate the list is to use a function based on the udf list plugin I have already written and look for UDFs which are at the right level to be converted to facts.

I have a couple of plugins for converting Generations files, but I suspect every Generations file will have subtly different problems.

The most recent file I did the first time I processed it had 60,000+ udf fields.

PeterR · Post by **PeterR** » 18 Aug 2011 16:40

Several invalid tags from various systems are listed in:
http://www.gencom.org.nz/GEDCOM_tags.html

Post by **tatewise** » 20 Aug 2011 00:34

I have decided to search every Fact for each UDF by using fhIsUDF().
Finding a UDF is OK, but converting it is trickier than expected, given that a text edit of the GEDCOM simply requires the UDF Tag to be changed to a valid Tag, and perhaps @@ replaced by @.

(1) DATE child tag
The value for this is a quoted string such as 'JUN 1927'.
So to create the DATE Object have used:

dtDate = fhNewDate(1999)
bResult = dtDate:SetValueAsText(strVal,true)
ptrNew = fhCreateItem(strTag,ptrTarget)
fhSetValueAsDate(ptrNew,dtDate)

(2) SOUR child tag
The value for this is a quoted string such as '@S99@' or maybe '@@S99@@'.
So to create the SOUR Lonk have used:

strVal = string.gsub(strVal,'[@,S]','')
ptrSour = fhNewItemPtr()
ptrSour:MoveToRecordById(strTag,tonumber(strVal))
ptrNew = fhCreateItem(strTag,ptrTarget)
fhSetValueAsLink(ptrNew,ptrSour)

(3) FAMC, FAMS, OBJE child tags
Presumably will have to do the same as SOUR for these too.

(4) NOTE child tag
The local NOTE tag has to change to NOTE2.
Have yet to investigate NOTE record links.

(5) Cannot create Custom Fact undefined in Fact Set
What I wanted to do was to use something like:

ptrNew = fhCreateItem('EVEN-BAP',ptrRecord) where BAP is the UDF Fact Tag, to create:
1 EVEN
2 TYPE BAP
But that yields a null pointer and nothing is created in FH.
The workaround requires the Custom Event: BAP to be defined in a Fact Set, then it works OK.
The following also does not work:

ptrNew = fhCreateItem('EVEN',ptrRecord)
ptrTyp = fhCreateItem('TYPE',ptrNew)
fhSetValueAsText(ptrTyp,'BAP')

So currently I convert the UDF to a defined Custom Attribute: UDF and add the UDF details as its value:

ptrNew = fhCreateItem('_ATTR-UDF',ptrRecord)
strTag = fhGetTag(ptrOld) -- UDF Tag name such as BAP
strVal = fhGetValueAsText(ptrOld)
if strVal ~= '' then
strVal = ' '..strVal -- UDF Tag attribute value
end
-- Add UDF Tag name & value as _ATTR-UDF attribute value
fhSetValueAsText(ptrNew,strTag..strVal)

Any suggestions welcomed...

YouTube · Post by **Jane** » 20 Aug 2011 08:08

1. Have you tried using: fhGetValueAsDate

2. Yes

3. Yes

4. Yes

5. To create new Fact Types use fhGetFactTag with the Create if none option.

What I have been doing for Generations is to use a two pronged attach.

1. Scan the file for UDFs and make a list
2. Use Lua directly on the gedcom to convert them over.

Post by **tatewise** » 21 Aug 2011 00:32

(1) DATE
Yes, tried fhGetValueAsDate() but fails because the value is a Text String not a Date Object.
bResult = dtDate:SetValueAsText(strVal,true) works well if the value Text String is a valid date.
But if not valid, the bResult is always false, even though the 2nd parameter is true to AllowPhrase!!!

(2) SOUR & (3) OBJE & (4) NOTE
These all convert links such as @@S99@@ or @@N99@@ correctly.
If the value is not a link, then the Tag is changed to SOUR2, OBJE2, NOTE2 for a text value.

(5) Create Custom Fact
Yes fhGetFactTag() should solve that problem.
However, I came to a better solution similar to your Generations technique:
Scan all Facts using fhIsUDF() == true to find UDF.
Create a dummy Fact Set within the LUA GUI database comprising all the UDF Facts.
Since I already load all the Custom Fact Sets it is then easy to select any UDF Fact and choose the Standard Fact or Custom Fact to convert to.

(6) EVEN & QUAY
These also need unique code to convert encoded Text Strings into Text Strings acceptable to FH API.

The Plugin is now in its final stages and should be ready to upload soon for Beta Testing by others.

Post by **tatewise** » 22 Aug 2011 19:12

While scanning all Facts for UDF the plugin also scans for any Facts that are not defined in any Custom Fact Set and places them in a dummy Fact Set within the LUA GUI database named Custom Facts.

PeterR has sent a GEDCOM with UDF examples that I am working on, although most of the UDF are converted OK already.
The main scenario not handled is SOURce Records with UDF sub-Tags - but I did not set out to convert them - perhaps in a later version.
If anyone has other GEDCOM with UDF Facts then I would love to try them out.

I the meantime Change Any Fact Tag V1.0 Beta has been added to the Plugin Store and I would welcome any feedback on how it performs.

It should preserve settings across PC Platforms, and cope with both V4 Projects and standalone GEDCOM.

Post by **tatewise** » 29 Aug 2011 00:00

An updated Change Any Fact Tag V1.1 has been added to the Plugin Store, which has many minor enhancements.
It now copes with many more UDF tag structures, and any untranslatable code is reported, and logged, and added as a SOURce Note to the Event or Attribute.
In fact SOURce Notes are extensively used for anything that won't fit anywhere else.
e.g.
If an Attribute is converted to an Event, then the Attribute Value is transferred to a SOURce Note.
If any Standard Fact has a TYPE Descriptor, then this is transferred to a SOURce Note to avoid any conflict with Custom Facts.

There are a couple of FH 'bugs' that need fixing, for which I have workarounds.
I also have developed a test suite, so that any new coding for extra tag structures can be quickly regression tested, to ensure the changes have not upset any existing features.

Post by **tatewise** » 11 Dec 2011 20:32

All the issues discussed above have either been resolved or are covered by the following threads:

Plugin: fhGetDisplayText function at
http://www.fhug.org.uk/cgi-bin/index.cg ... y&num=5255

FH API fhSetValue... ADDR Problem at
http://www.fhug.org.uk/cgi-bin/index.cg ... y&num=5410

Date:SetValueAsText and Date Phrases at
http://www.fhug.org.uk/cgi-bin/index.cg ... y&num=5403

Family Historian User Group