* Build Tree From Raw Census Data plugin for one-name study

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
User avatar
mikej
Gold
Posts: 12
Joined: 07 Jun 2011 10:06
Family Historian: V7
Location: Wales

Build Tree From Raw Census Data plugin for one-name study

Post by mikej » 31 Aug 2013 18:30

This plugin is based on Jane Taubman's “Build Tree From CSV File” plugin.
It takes a formatted and sorted CSV file containing raw census household data
and builds a tree consisting of parent(s) and children in each household.
All other persons in the household are linked to the head as an Associated Person.
A citation is attached to each event pointing to the census source with PRO reference.

It is assumed that the person enumerated as the wife of the head of household
is the mother of all the hoh's children.

The plugin is designed for use in a one-name study and assumes,
at present, only one household per census page.

The Associated person section needs more work.
Maybe there needs a progress bar or a please wait message as it takes several seconds
to import 1200+ persons in my tests.

At present all sorting is done externally before running the plugin.
It is hoped to include this sorting within the plugin eventually.

The csv file requires the following columns (this may change):
ID,Forenames,Surname,Collections From,Sex,Country,Category,
Record Collection,Record Set,County,Year,BirthDate,
Age,Street,Occupation,Relationship To Head Of Household,
Birth Place,Birth Town,Birth County,City,House Name,
Town,Condition,Archive Reference,Folio,Page,Piece Number,
Other Household Members,House Number

The csv file needs to be sorted by several columns to enable each household to be read and processed in sequence.
The sort sequence required is:
Archive Reference, Piece Number, Folio, Page, in ascending order
And Relationship To Head Of Household in the custom sort order of:
Head
Wife
Son
Daughter

The ID is then assigned in ascending order to the whole file.

I am sure it will not win any awards for programming style, but it hopefully does the job.

I would be interested to know what others think.
Mike
Attachments
Build Tree From Raw Census Data.fh_lua
(23.33 KiB) Downloaded 212 times
Last edited by mikej on 31 Aug 2013 19:15, edited 2 times in total.

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by Jane » 31 Aug 2013 19:05

Add Assessment to Citations for a Source.fh_lua
(5.46 KiB) Downloaded 244 times
Strange - it just let me upload this one.

Can you check the extension was fh_lua and was all in lowercase.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

User avatar
mikej
Gold
Posts: 12
Joined: 07 Jun 2011 10:06
Family Historian: V7
Location: Wales

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by mikej » 31 Aug 2013 19:14

That was it.
Thanks Jane.
Regards
MikeJ

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by Jane » 31 Aug 2013 22:23

Do you create the original source outside of FH? I don't think I have seen it use uppercase for the extension.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

User avatar
mikej
Gold
Posts: 12
Joined: 07 Jun 2011 10:06
Family Historian: V7
Location: Wales

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by mikej » 01 Sep 2013 06:44

I wrote the plugin within Family Historian.
I wasn't sure where FH saved my plugin so I pasted the text into Notepad and saved from there.
It was at this point I had confusion over the extension and missed out the "fh_" bit. My error.
Regards
MikeJ

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by Jane » 01 Sep 2013 08:07

The best way to get the plugins out, is to use the "Export" button on the Plugin in Window
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

User avatar
mikej
Gold
Posts: 12
Joined: 07 Jun 2011 10:06
Family Historian: V7
Location: Wales

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by mikej » 12 Sep 2013 12:21

A new version of this plugin is attached.
Flags are attached to individuals to highlight the need for closer attention.
A progress bar and end of import status message have been added.
Details are in the source.

mikej
Attachments
Build Tree From Raw Census Data.fh_lua
(30.92 KiB) Downloaded 206 times
Regards
MikeJ

User avatar
Jane
Site Admin
Posts: 8441
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by Jane » 12 Sep 2013 12:34

I took a quick look at the code, one comment. The main loop uses pairs, but you look to be keying the table on a number and pairs does not guarantee key order, so you might want to use ipairs to ensure the table is processed in arrival order.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."

User avatar
mikej
Gold
Posts: 12
Joined: 07 Jun 2011 10:06
Family Historian: V7
Location: Wales

Re: Build Tree From Raw Census Data plugin for one-name stud

Post by mikej » 12 Sep 2013 16:29

Thanks Jane.
Regards
MikeJ

Post Reply