* Country Summary Help

[British Kiwi]

Good evening, from wintery Melbourne.

I have been having a little play with adding some icons to my diagrams and decided I wanted to get a summary of what countries my ancestors were born and died in. After coming a little short in queries, I decided to finally jump into writing a plugin. I used to make MS Access programs in another life with VBA, so have some basic programming skills (I know my IT colleagues scoffed at VBA ) anyway, I am a little rusty.

I managed to write a little plugin which told me how many were born in each country. I used the sample database which was a little annoying as the places aren't consistent. All my places in my main database have 4 parts, but I should easily be able to tweak before I use it on my main file. I can also change it to do the same for death countries, however, my problem is having a table/grid combining the two, eg:

Code: Select all

Country     No of Births  No of Deaths
Australia   1                      0
England    50                    45

etc etc.

I tried to tweak what I did, but that didn't quite work. I have also had a look at the Jane's code for the Surname Summary Report, but I can't get that to work either. I think the way I would have approached it with Access/VBA is slightly different.

This is the code I wrote based on one of the examples which just deals with one type:

Code: Select all

	tblCountryOfBirths = {}		-- Define array for Countries of Birth
	pi = fhNewItemPtr()	-- declare pointer
	pi:MoveToFirstRecord("INDI")	-- point to the first individual record
	while not pi:IsNull() do
		-- For each Person Add the Country of Birth to the list
		strPlace = fhGetItemText(pi,'%INDI.BIRT[1].PLAC>%')
		strCountry = fhCallBuiltInFunction("TextPart",strPlace,3,0,"STD")
		if (tblCountryOfBirths[strCountry] == nil) then
			tblCountryOfBirths[strCountry] = 1
		else
			tblCountryOfBirths[strCountry] = tblCountryOfBirths[strCountry] + 1
		end
		pi:MoveNext()
	end
	-- Build Tables for the result set columns for Country and Qty
	tblCountryOfBirth = {}
	tblcount={}
	ii = 0 -- line counter for result tables
	for strCountry, iQty in pairs(tblCountryOfBirths) do
		ii = ii + 1
		tblCountryOfBirth[ii] = strCountry
		tblcount[ii] = iQty
	end
	fhOutputResultSetColumn('Country of Birth', 'text',tblCountryOfBirth,ii,80,'align_left',2,true)
	fhOutputResultSetColumn('Count', 'integer', tblcount,ii,40,'align_right',1,false)

So I suppose I have a couple of questions which I think might help me navigate this:
1. When creating a table to store the information, do I need to do one for each column i.e. one for storing the number of births by country and a second one for storing the number of deaths? I thought that they would act like an array, but getting myself very confused.

2. In my head I thought I would have three columns and depending on what I was counting it would just update the relevant column, so find the country and +1 to the column.

Any pointers would be greatly appreciated it. I reckon if I can solve this problem, I might be able to do all sorts of things.

Thanks in advance!

[ColeValleyGirl]

Lua tables can be all things to all people.

I might create a table of tables, indexed on country. I'd probably use a key other than an integer because i think it makes things more understandable (and you don't need to remember which entry is the birth and which it the death. (But if you use a key, you have to use pairs to iterate over the table, not ipairs which only works for indexed tables.)

So something like

Code: Select all

Countries = {
England = {Births = 5, Deaths =2},
Wales = {Births = 7, Deaths = 0}
}

So you'd access a value by e.g. Countries["England"]["Deaths"]

But there's a lot of personal preferences involved here.

Incidentally TextPart will take -1 as the second parameter to return the last part of a comma-separated string. As long as your places all end with Country, the number of columns won't matter.

[British Kiwi]

Thanks for that, I will see how I go with that. I found another topic through the help file all about Tables which is great evening reading

The tip about TextPart has made my day! I was going in circles in a query until I found that function, and now you've made it much easier!

Can I ask a question about what I originally wrote (totally plagiarised of course). The line

tblCountryOfBirths[strCountry] = tblCountryOfBirths[strCountry] + 1

I know what it is doing and why, but was wondering what the table would look at this point. So if strCountry is England and previously there was 21 entries, would this row would have two columns "England, 22" or would it just be a number? it is just weird that there isn't two variables the name of the country and the count. Not sure if I'm making sense.

[ColeValleyGirl]

Code: Select all

tblCountryOfBirths[strCountry] = tblCountryOfBirths[strCountry] + 1

would yield a single cell in the row tblCountryofBirths[England] with a numerical value.

Your actual table would be e.g.

Code: Select all

tblCountryOfBirths = {
England = 22, Wales = 17
} 

and the value of tblCountryoFBirths["England"] is 22

[Mark1834]

You can view the full contents of any table in the debugger window while the plugin is paused (double click on the table name). I find that very useful to make sure that the table actually contains what I think it contains!

[Mark1834]

Incidentally, tblCountryofBirth[strCountry] = foo can also be expressed as tblCountryofBirth.strCountry = foo. That version is common in many standard libraries and store plugins, so you need to be familiar with it even if you don’t use it yourself.

[British Kiwi]

Thanks Mark, that helps a bit. I was stepping through, so could check the variables, but couldn't see the table.

I'm going to have a little play and then I will be back (though struggling to stay awake so if I disappear I've given up for the night!

[British Kiwi]

Okay, so switched some of the syntax to:

if (tblCountryOfBirths.strCountry == nil) then
tblCountryOfBirths.strCountry = 1
else
tblCountryOfBirths.strCountry = tblCountryOfBirths.strCountry + 1
end


Code works without errors, however, now the results are just one record
Country of Birth Count
strCountry 104


So how do I get it to say

tblCountryOfBirths.England or tblCountryOfBirths.Wales etc?

[tatewise]

Sorry to jump in but tblCountryOfBirths[strCountry] is NOT the same as tblCountryOfBirths.strCountry

Assuming strCountry is "England" then tblCountryOfBirths[strCountry] is the same as tblCountryOfBirths.England

[Mark1834]

Yes, I didn't make that clear - the item following the dot is the value of a string table element ('England', 'Australia', etc), not the name of the variable. It also cannot contain spaces, so tblCountry.Australia is fine, but not tblCountry.New Zealand.

[British Kiwi]

Okay great, that makes sense.
So let's say everyone has a birth place and a death place for ease. My final table is going to be:
Country No of Births No of Deaths
England 25 12
Scotland 12 10

Etc

If I loop through each INDI and assign the country of birth and death to strBirthCountry and strDeathCountry respectively, how do I assign the count. Do I need a separate table for each type first before combining? Or can I use:
tblCountry[strBirthCountry] and tblCountry[strDeathCountry] to find the row where the index is England for example and then how do I assign the count for the relevant column?

Also Mark, I'm struggling to double click on the table to see the contents, it isn't listed in the variables in debugging. It does pop out on my results.

[Mark1834]

Regard tables as collections of key/value pairs. So for a table of birth countries, the key would be the country name and the value would be the number of births in that country.

So tblCountryOfBirth would contain two rows, England => 25 and Scotland => 12, in your example, and tblCountryOfDeath would also contain two rows, England =>12 and Scotland => 10.

There is one "gotcha" to consider - you will have to initialise the table values first, otherwise you will be adding 1 to nil, which is not permitted.

Your loop will look something like

Code: Select all

a line to set strCountryOfBirth to the birth country
if not tblCountryOfBirth[strCountryOfBirth] then tblCountryOfBirth[strCountryOfBirth] = 0 end  -- initialise to 0 if not yet done
tblCountryOfBirth[strCountryOfBirth] = tblCountryOfBirth[strCountryOfBirth] + 1
repeat for deaths

That's a mix of description and actual code, but I hope it makes sense!

Sometimes it is useful to insert a "dummy" line that doesn't do anything useful before the end of your plugin, so you can pause it there to see the final values of your variables before execution ends. I generally use

Code: Select all

_ = true

where the single underscore is by convention a "dummy" placeholder for a variable that is not used for anything.

[British Kiwi]

Okay that makes sense.
So the next trick is to put the two (or more) together. So would you loop through tblBirthCountry and then find the result in tblDeathCountry?

And what happens if one country doesn't appear in both tables e g. I have a great aunt born in Crete and no one died there. Is it safer to have a master tblCountry which will have a distinct list of countries to then use to find the results if they exist.

Sorry for the step by step questions.

[ColeValleyGirl]

In the plugin window, if your table is a global table, you need to go Debug > Options and enable the display of global tables.

As you seem to have plumped for two parallel tables, rather thana table of tables, I'll bow out of the other discussion rather than confuse you.

[British Kiwi]

Okay I'll try that when I get home.

A table of tables sounds good too. Maybe I will continue how I'm going and then investigate the table of tables. I have found a lot of lua you tube videos which I might check out on the train. Maybe an FH expert might like to do one hint, hint

[Mark1834]

IMO, two parallel tables is the best option for developing your first working draft, so you can concentrate on the basic principles of how tables are structured and manipulated. However, it is not the best option for creating a single summary table of the results. Here, a table of tables, with the country as the key and the value as a table such as Births => 23, Deaths => 12 helps keep everything together, which you will need for the final step of copying the data into a new set of tables for the final output display.

It's a lot of new concepts for just one small table of output, but the general principles will be good background for future plugins.

[tatewise]

To make a global table visible in the debugger you can use the Debug > Options and enable the display of global tables as suggested by Helen.

Alternatively, declare the table as local even at the outermost scope and it is visible in the debugger by default.
e.g.
local tblCountryOfBirth = {}

An alternative technique for ensuring a table key exists is:
tblCountryOfBirth[strCountryOfBirth] = ( tblCountryOfBirth[strCountryOfBirth] or 0 ) + 1

That relies on the logic where if tblCountryOfBirth[strCountryOfBirth] does not exist it is false so the or value 0 used.
Otherwise, it is true and its numerical value is used.

[British Kiwi]

Thanks. I realised how much I enjoy programming (not enough to go back to it though, although it was only ever part of my job), so this has been fun. So, I finally have a combined table, I think i need to add a few "local" added and need to declare some variables, but this is what I have:

Code: Select all

--[[
@Title: Country of Birth Summary Report
@Author: Melanie Marsh
@LastUpdated: June 2022
@Description: Counts and Lists All Countries of Birth in the File
]]

	tblMasterCountries = {}		-- Definne array of master list of countries
	tblBirthCountries = {}		-- Define array for Countries of Birth
	tblDeathCountries = {}    -- Define array for Countries of Death

	pi = fhNewItemPtr()	-- declare pointer
	pi:MoveToFirstRecord("INDI")	-- point to the first individual record
	
	while not pi:IsNull() do
	

		-- For each Person get the Country of Birth
		strBirthPlace = fhGetItemText(pi,'%INDI.BIRT[1].PLAC>%')
		strBirthCountry = fhCallBuiltInFunction("TextPart",strBirthPlace,-1)

		if strBirthCountry == "" then
			strBirthCountry = "Not Set"
		end

		-- For each Person Get the Country of Death
		strDeathPlace = fhGetItemText(pi,'%INDI.DEAT[1].PLAC>%')
		strDeathCountry = fhCallBuiltInFunction("TextPart",strDeathPlace,-1)
		
		if strDeathCountry == "" then
			strDeathCountry = "Not Set"
		end

		-- Add Birth Country to the Master List
		if (tblMasterCountries[strBirthCountry] == nil) then
			tblMasterCountries[strBirthCountry] = 1
		else
			tblMasterCountries[strBirthCountry] = tblMasterCountries[strBirthCountry] + 1
		end

		-- Add Death Country to the Master List
		if (tblMasterCountries[strDeathCountry] == nil) then
			tblMasterCountries[strDeathCountry] = 1
		else
			tblMasterCountries[strDeathCountry] = tblMasterCountries[strDeathCountry] + 1
		end
	

		-- Add the Country of Death to the Birth List
		if (tblBirthCountries[strBirthCountry] == nil) then
			tblBirthCountries[strBirthCountry] = 1
		else
			tblBirthCountries[strBirthCountry] = tblBirthCountries[strBirthCountry] + 1
		end
	

		-- Add the Country of Death to the Death List
		if (tblDeathCountries[strDeathCountry] == nil) then
			tblDeathCountries[strDeathCountry] = 1
		else
			tblDeathCountries[strDeathCountry] = tblDeathCountries[strDeathCountry] + 1
		end

	pi:MoveNext()


	end


	-- Build Tables for the result set columns for Country and Qty
	tblMasterCountry = {}
	tblBirthCount={}
	tblDeathCount = {}

	ii = 0 -- line counter for result tables

	for strMasterCountry, iMasterQty in pairs(tblMasterCountries) do

		ii = ii + 1

		tblMasterCountry[ii] = strMasterCountry

		if (tblBirthCountries[strMasterCountry] == nil) then
			tblBirthCount[ii] = 0
		else
			tblBirthCount[ii] = tblBirthCountries[strMasterCountry]
		end


		if (tblDeathCountries[strMasterCountry]== nil) then
			tblDeathCount[ii] = 0
		else
			tblDeathCount[ii] = tblDeathCountries[strMasterCountry]
		end


	end


	fhOutputResultSetColumn('Country', 'text',tblMasterCountry,ii,80,'align_left',2,true)
	fhOutputResultSetColumn('No of Births', 'integer', tblBirthCount,ii,40,'align_right',1,false)
	fhOutputResultSetColumn('No of Deaths', 'integer', tblDeathCount,ii,40,'align_right',1,false)

I'm actually not too sure what method I have done, but tomorrow, I am going to try and streamline it a bit better.

Any suggestions will be gladly welcome.

[tatewise]

Here are a few tips:

Use Edit > Insert Script Header as there are some significant fields missing.

Use while pi:IsNotNull() do which is a little neater.

Instead of "Not Set" use such as "~ Not Set ~" or such like so that if the Result Set is sorted by Country it does not get sorted in with Norway but at the end or beginning.

Instead of:

Code: Select all

	for strMasterCountry, iMasterQty in pairs(tblMasterCountries) do
		ii = ii + 1

		tblMasterCountry[ii] = strMasterCountry

		if (tblBirthCountries[strMasterCountry] == nil) then
			tblBirthCount[ii] = 0
		else
			tblBirthCount[ii] = tblBirthCountries[strMasterCountry]
		end
		
		if (tblDeathCountries[strMasterCountry]== nil) then
			tblDeathCount[ii] = 0
		else
			tblDeathCount[ii] = tblDeathCountries[strMasterCountry]
		end

This is an alternative using the table library and or operator:

Code: Select all

	for strMasterCountry, _ in pairs(tblMasterCountries) do
		table.insert( tblMasterCountry, strMasterCountry )
		table.insert( tblBirthCount, tblBirthCountries[strMasterCountry] or 0 )
		table.insert( tblDeathCount, tblDeathCountries[strMasterCountry] or 0 )
	end
	local ii = #tblMasterCountry

[Mark1834]

Excellent, Melanie - I've run it on a copy of my main project and even in the debugger it runs to completion in a fraction of a second and gives a believable output (my informal test - it looks about right....).

The code is slightly repetitive in places, and the more experienced Lua coder could probably cut out some of the intermediate steps, but in this context, that doesn't matter in the slightest. It is clear and understandable and gets the job done. That's far more important than shaving a few milliseconds off the run time with a more "elegant" solution. You can move on to those as you firm up your understanding of the basics.

Some minor comments/suggestions I jotted down:

• pi is valid in Lua as a variable name, but I think we should avoid it because of its mathematical meaning. I generally use pI, pF, etc.
• The [1] is superfluous if you only want the first value. Leave it out and it defaults to [1].
• You have used while not pi:IsNull() do as your loop condition. It's probably clearer to express this as while pi:IsNotNull() do. For some reason, CP use the uglier form in the plugin help, so I started with that as well!
• You have noticed that the empty string is NOT evaluated as false in Lua. I think it is in VBA and some other languages, so that can trip people up.
• if foo == nil then can often be expressed more clearly as if not foo then...
• You can use table.insert(table name, value) to add values to tables where the keys are continuous integers, and then the #table value to return the current size of a table. Most Lua coders would do it that way rather than use a separate counter.
• For non-trivial plugins, I always create a main() function and put the code in there, so the global section of the code may be no more than a single line calling main(). It keeps all variables local (define them with a local prefix), which is usually regarded as best coding practice, as well as making them easier to examine in the debugger, which shows only variables in scope at the point the script is paused. Picking up one of my comments from yesterday, I often put a dummy line in the global code after the call to main() so I can pause the script before exiting and check for any stray global variables where I have omitted the local keyword in the definition (Lua defaults to global if local is not specified). Unlike C, main() is not a required name - you can call it whatever you want in Lua, main() is just a convention.
• CP often use semi-colons to mark the end of lines. They are optional in Lua, and most plugin authors don't bother with them (personally, I can't see the point of them, but it you trained on a language that requires them, it's probably just habit!).

[British Kiwi]

Thanks to everyone that's helped. I was so pleased it output what I wanted. I wasn't too keen on the repetitive code so the tips from Mike and Mark I will try tonight. I like to try and stick to standard conventions but I haven't worked out what they are but your feedback should help. I'm back on the train now so will have to wait to try it out.

[Mark1834]

There isn’t a definitive guide for Lua, but this document is the guide for Python, which has a lot of similarities to Lua.

It’s well written and worth a skim read to pick up some of the general principles.

[ColeValleyGirl]

And of course there's Programming in Lua, if you haven't already found it. Online free for Lua 5.1, but if you're going to do more than dabble (and learn by reading rather than watching or trying, say) I suggest buying the latest version (4th edition).

[tatewise]

Have you found the FHUG KB Getting Started Writing Plugins that has much advice and cross-references to resources.

[British Kiwi]

Thanks Mark and Helen. I will plow through those. I have tomorrow off so will sneak some reading time.

I have managed to incorporate some of the tips and suggestions and it is looking better. However, for some reason I am having difficulties calling functions which is a little weird. I had a look at a published plugin and I can't work out what is wrong. I ended up creating a basic code to see if I can get it to work. I wrote this:

Code: Select all

 function main()

 	local strPlace
	strPlace = "Cumbria,England"
	strBirthCountry = GetCountry(strPlace)
	print(strBirthCountry)

 end

 --additional function

 function getCountry(strPlace)

	strCountry = fhCallBuiltInFunction("TextPart",strPlace,-1)
	return strCountry

 end

I was hoping to pass the birth/death place to a function that would pull out the country and/set it to ~Not Set~ (I plan eventually to separate those that are not known and those who are not dead etc. Anyway when I step into this code it starts on line 8, then line 1, then line 17, line 12, then line 17 again! I added function main() end around my other plugin and that does similar too. I had a look at some of the information you both sent and some videos I found and I cannot what I have done wrong?

This bit has me totally baffled!