Page 1 of 1
Census DEA - accent characters error?
Posted: 09 Oct 2021 04:57
by sbell95
I have just begun to encounter the following error after running Helen's DEA 'Record Census Data (UK)' v1.4:
- Screenshot 2021-10-09 155057.jpg (18.07 KiB) Viewed 1649 times
This is despite not having any characters out of the ordinary in any of my text fields?
- Screenshot 2021-10-09 155115.jpg (54.05 KiB) Viewed 1649 times
I also note that this error does not seem to be able to be repeated in the sample project... but doesn't seem to otherwise cause any issues in my main project as the plugin still runs and functions as expected.
Any ideas?
Re: Census DEA - accent characters error?
Posted: 09 Oct 2021 08:56
by tatewise
This is related to handling the innumerable UTF-8 multi-byte Unicode characters beyond the one byte ANSI characters.
That includes not only accented foreign language characters but all Asian characters and unusual symbols.
Plugins should be designed to handle all UTF-8 Unicode characters.
However, the 'Record Census Data (UK)' and related DEA have their File > Encoding set to ANSI and not UTF-8.
So if there is nothing in the Plugin script to counter that, then the warning you are seeing is automatically produced regardless of whether any Unicode characters are actually processed or not.
If you were to add some accented foreign language characters to your data then they would probably get mishandled.
I am surprised the warning does not occur in the Sample Project as it should be the same for all Projects.
Re: Census DEA - accent characters error?
Posted: 09 Oct 2021 09:21
by ColeValleyGirl
I'm surprised -- I'm not seeing that behaviour.
The DEAs all invoke setIupDefaults from the fhUtils library, which sets string encoding to UTF8.
Code: Select all
local function setIupDefaults()
local function getRegKey(key)
local sh = luacom.CreateObject("WScript.Shell")
local ans
if pcall(function()
ans = sh:RegRead(key)
end) then
return ans
else
return nil, true
end
end
iup.SetGlobal("CUSTOMQUITMESSAGE", "YES")
-- Get Font and size from the Registry for the property box and use as the default font.
local v
-- Check for preview value and use if existing
v = getRegKey("HKEY_CURRENT_USER\\Software\\Calico Pie\\Family Historian 7 Beta\\2.0\\Preferences\\PDX Font")
if not v then
v = getRegKey("HKEY_CURRENT_USER\\Software\\Calico Pie\\Family Historian\\2.0\\Preferences\\PDX Font")
end
local t = { stringx.splitv(v, ",") } -- first value is size and 14th is font name
iup.SetGlobal("DEFAULTFONT", t[14] .. " " .. t[1] / 20)
iup.SetGlobal("UTF8MODE", "YES")
iup.SetGlobal("UTF8MODE_FILE", "NO") --use file names in the current locale
fhSetStringEncoding("UTF-8")
end
Sarah, what encoding is your project saved in (Tools > Preferences > File Load/Save > File Saving Options)? Is it UTF8 (default) or UTF16?
Updated to add: changing save enconding to utf16 doesn't make any difference -- I still can't reproduce the error.
Re: Census DEA - accent characters error?
Posted: 16 Dec 2021 10:54
by ColeValleyGirl
This was fixed in the most recent release of the DEA.
P.S. Mike, it was nothing to do with the DEA encoding.