Page 1 of 1

Build Tree From Raw Census Data plugin for one-name study

Posted: 31 Aug 2013 18:30
by mikej
This plugin is based on Jane Taubman's “Build Tree From CSV File” plugin.
It takes a formatted and sorted CSV file containing raw census household data
and builds a tree consisting of parent(s) and children in each household.
All other persons in the household are linked to the head as an Associated Person.
A citation is attached to each event pointing to the census source with PRO reference.

It is assumed that the person enumerated as the wife of the head of household
is the mother of all the hoh's children.

The plugin is designed for use in a one-name study and assumes,
at present, only one household per census page.

The Associated person section needs more work.
Maybe there needs a progress bar or a please wait message as it takes several seconds
to import 1200+ persons in my tests.

At present all sorting is done externally before running the plugin.
It is hoped to include this sorting within the plugin eventually.

The csv file requires the following columns (this may change):
ID,Forenames,Surname,Collections From,Sex,Country,Category,
Record Collection,Record Set,County,Year,BirthDate,
Age,Street,Occupation,Relationship To Head Of Household,
Birth Place,Birth Town,Birth County,City,House Name,
Town,Condition,Archive Reference,Folio,Page,Piece Number,
Other Household Members,House Number

The csv file needs to be sorted by several columns to enable each household to be read and processed in sequence.
The sort sequence required is:
Archive Reference, Piece Number, Folio, Page, in ascending order
And Relationship To Head Of Household in the custom sort order of:
Head
Wife
Son
Daughter

The ID is then assigned in ascending order to the whole file.

I am sure it will not win any awards for programming style, but it hopefully does the job.

I would be interested to know what others think.
Mike

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 31 Aug 2013 19:05
by Jane
Add Assessment to Citations for a Source.fh_lua
(5.46 KiB) Downloaded 245 times
Strange - it just let me upload this one.

Can you check the extension was fh_lua and was all in lowercase.

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 31 Aug 2013 19:14
by mikej
That was it.
Thanks Jane.

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 31 Aug 2013 22:23
by Jane
Do you create the original source outside of FH? I don't think I have seen it use uppercase for the extension.

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 01 Sep 2013 06:44
by mikej
I wrote the plugin within Family Historian.
I wasn't sure where FH saved my plugin so I pasted the text into Notepad and saved from there.
It was at this point I had confusion over the extension and missed out the "fh_" bit. My error.

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 01 Sep 2013 08:07
by Jane
The best way to get the plugins out, is to use the "Export" button on the Plugin in Window

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 12 Sep 2013 12:21
by mikej
A new version of this plugin is attached.
Flags are attached to individuals to highlight the need for closer attention.
A progress bar and end of import status message have been added.
Details are in the source.

mikej

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 12 Sep 2013 12:34
by Jane
I took a quick look at the code, one comment. The main loop uses pairs, but you look to be keying the table on a number and pairs does not guarantee key order, so you might want to use ipairs to ensure the table is processed in arrival order.

Re: Build Tree From Raw Census Data plugin for one-name stud

Posted: 12 Sep 2013 16:29
by mikej
Thanks Jane.