* utf8 library error in Lua 5.3

[tatewise]

In the Zooming Media images (20335) thread I have updated the Rename Media to Match Title Field plugin.
I tried using the utf8.gsub(...) function to ensure Media Titles with Unicode UTF-8 characters were fully supported.
However, when looping through the Media records it would crash after a dozen or so loops.
So I have reverted to string.gsub(...) which will probably even handle Unicode UTF-8 characters I hope.

I narrowed the cause of the problem down to the FH pointer 'userdata' structure.
The following minimal script illustrates the problem and can be Run normally or in debug mode:

Code: Select all

utf8 = require(".utf8"):init()
for i = 0, 99 do
	item = fhNewItemPtr()		-- userdata causes \utf8\context\runtime.lua:30 failure
	text = utf8.gsub("ABCD","X"," ")
end

Usually a short loop such as for i = 0, 5 do does not fail. Only greater repetitions raise the following error:

Code: Select all

An error has occurred - plugin failed to complete
[string "X"]:37: error in __gc metamethod (...)\Family Historian\Program\Lua\\utf8\context\runtime.lua:30: System error)!
No changes have been made to data records.

Several other forms of FH pointer generating API functions have the same effect.

The specific utf8.gsub parameters don't appear to be relevant.

Line 30 in the \utf8\context\runtime.lua file involves a complex setmetatable(...) function.

The problem only affects FH V7 Lua 5.3 and does NOT affect FH V6 Lua 5.1 at all.
Can others repeat the same symptoms?
Should it be reported to CP? Perhaps with suitable debug tools, the problematic interaction can be diagnosed?
I'm not sure reporting it to the utf8 library author will help unless we can find a non-FH method of getting the error.

[ADC65]

Yes, I get exactly the same error when I run that code.

The loop count at which it fails changes with each run. Sometimes it will reach 30, sometimes it will not reach 15.

[ColeValleyGirl]

I doubt there's much mileage in asking CP to investigate.

It feels like a resource problem or perhaps garbage collection?

[tatewise]

I don't understand what sort of resource or memory problem it could be.
The script is tiny and usually fails within 30 loops, so not much resource or memory can get used.

If the item = fhNewItemPtr() statement is removed then it runs perfectly.
So it looks like some part of that FH fhNewItemPtr() implementation is the cause.

Anyway, I have tried aggressive memory management as shown below, and it fails just the same.
If anything it made things worse by often failing on the first loop.

Code: Select all

local intPause = collectgarbage("setpause",50)
local intStep = collectgarbage("setstepmul",300)

Can you give some clue to what sort of resource problem might be involved?

[ColeValleyGirl]

Not really, but since the only thing the

Code: Select all

item = fhNewItemPtr() 

line does is create a variable that is never used, how can it be interacting with the utf8 library except by using resources?

[tatewise]

__gc metamethod(...) is mentioned in the error message (whatever that is).
The setmetatable(...) function is on line 30 of the \utf8\context\runtime.lua file.
So I wonder if fhNewItemPtr() somehow upsets those 'resources' but only CP would know.

[ColeValleyGirl]

__gc metamethod(...) is mentioned in the error message (whatever that is).

Objects in lua are garbage collected. Sometimes, you need to free some resource, want to print a message or do something else when an object is destroyed (collected). For this, you can use the __gc metamethod, which gets called with the object as argument when the object is destroyed. You could see this metamethod as a sort of destructor.

The result of simple search

Hence me thinking 'garbage collection'.

That metatable operation on the line where utf8 errors doesn't look particularly complex -- it's just defining the metatable for an object (which is not the object created by item).

[ColeValleyGirl]

Another reason to think it's related to garbage collection is the fact that it runs OK in Lua 5.1, and the Lua's garbage collection changed between 5.1 and 5.3

[tatewise]

I've spent all morning trying to understand how __gc metamethod impacts the utf8 runtime.lua file without progress.

Adjusting the collectgarbage(...) settings can increase or reduce the probability of failure without correlation to the settings.

The only two things that guarantee success are collectgarbage("stop") or removal of fhNewItemPtr().

[ColeValleyGirl]

This also works:

Code: Select all

utf8 = require(".utf8"):init()
for i = 0, 99 do
        collectgarbage()
	item = fhNewItemPtr() 
	text = utf8.gsub("ABCD","X"," ")
end

[tatewise]

Thank you, that is worth knowing and may offer solutions in various scenarios.

[ColeValleyGirl]

I should credit Programming in Lua (Fourth edition).