Family Historian User Group

Posted: **15 Mar 2021 18:09**

Suppose I have a plugin (A) that does some things that require substantial FH function and data structure knowledge. Yet the overall function of the plugin also encompasses relatively easy (read straight forward Lua) code (B) that should typically be provided by the user (e.g. because of a specific user implementation or language pack dependent).

Is it possible to have plugin A and show the user a list of plugins (B) for plugin A and call B from within A? And if so, how?

Posted: **15 Mar 2021 18:20**

Yes and No.

An FH plugin cannot run another FH plugin that is in the Plugin Store or in the FH Plugins folder.

However, there is an Lua mechanism for running some Lua script loaded from a file.
See the load(...) and loadfile(...) functions.

There may be alternative solutions to the problem depending on the details.

Posted: **15 Mar 2021 18:27**

If you're planning a plugin for the plugin store, it has to be self contained apart from the libraries delivered with fh.

Except if you're going to ask users to provide a module customised to them. You would distribute A and specify the constraints on the module B. You would need to understand the format of lua modules so read up on that and have a look at the structure of th fhutils module.
By

Posted: **15 Mar 2021 21:41**

@Mike What details do you have in mind? I also thought about reading the user provided script from file, do some checking as to format and conventions etc. and then run it (preferably in a way that I can catch user errors; can be done in Lua, I hope?). Is that a viable course of action?

@Helen Yes, I am thinking of a plugin within the boundaries set bij CP (self contained) and have the user execute her/his local script within this plugin (as what Mike suggests). I am not completely clear on what you mean with the part about modules. I know about them but don't see how they come into this. Please explain some more, all hints tips and ideas are welcome before I start on this plugged-in plugin trail.

The basic concept is to have plugin A (in the store) run by a user. A then asks the user which of his local plugins (B, conforming to A's API) he wants to execute. Then A runs and calls B etc.). The major reason for this: hide complexity from the user but still allow him/her to do 'plugin like things' in Lua. Of course there may be a number of A plugins, each one specific for a class of tasks.

Posted: **15 Mar 2021 23:04**

The kind of details are the extensive options that I've built into my plugins to let users choose how they operate.

Have you read the load(...) and loadfile(...) definitions in the Lua Reference Guide?
They will only execute syntactically valid Lua script or return an error message.
You could load a file and check its contents in Plugin A before submission to the load(...) function.

Can you give some idea of the kind of Lua scripts a user might be capable of writing and how they would debug them.
I think it would only be a tiny proportion of users who would feel comfortable writing Lua.
Perhaps there is another way of providing the type of functionality you are considering.

Posted: **16 Mar 2021 09:44**

JoopvB wrote: ↑
15 Mar 2021 21:41
I am not completely clear on what you mean with the part about modules.

A module/library (B) is a 'black-box' set of functions -- or it could be a single function -- which can be invoked by plugin A, and then the functions invoked using B.function1, B.function2 etc. They can also contain data items e.g. tables which can be made directly available to the plugin if wanted and can run code immediately they're initialised.

You can 'require' a library in which case fh searches for it by library name in the following places (I required a library called x):

Code: Select all

	'C:\ProgramData\Calico Pie\Family Historian\Plugins\x.fh_lua'
	'C:\ProgramData\Calico Pie\Family Historian\Plugins\x.lua'
	'C:\Program Files (x86)\Family Historian\Program\Lua\x.lua'
	'C:\Program Files (x86)\Family Historian\Program\Lua\x.fh_lua'
	'C:\Program Files (x86)\Family Historian\Program\Lua\x53.dll'
	'C:\Program Files (x86)\Family Historian\Program\Lua\x.dll'
	'C:\ProgramData\Calico Pie\Family Historian\Plugins\x.dll'

Loadfile will work as well, but there are differences -- your best bet is to look at the online Lua documentation to understand those and decide what is best for your purpose.

Posted: **16 Mar 2021 10:33**

tatewise wrote: ↑
15 Mar 2021 23:04
I think it would only be a tiny proportion of users who would feel comfortable writing Lua.

Mike, you're in danger of making that a self-fulfilling prophecy by making it all sound hard and difficult.

Only a small number of people would perhaps want to take on the task of coding plugins/libraries as complex as yours, especially given the complexities of iup.

But asking them to code a table of data to be used by a plugin, or to amend an existing table, is perfectly possible (a number of people have asked me on the forums and off them how to amend the DEAs and found it surprisingly simple). I'd go so far as to say that anyone who can write a query can cope with a Lua table...

Within fhutils, getParam is data-table driven which makes iup much simpler to use; and there are other functions in there to reduce the complexity of coding. Plus, as you may remember, Shosh is intending to produce a beginners guide to writing plugins, which I am looking forward to.

There are already users (as you know) who are happy to write plugins for personal use; because they just get on with it, we're not aware of all of them. Also, time is on our side -- users coming to FH now are more likely to have IT experience, and be more confident with getting to grips with new technology. What they don't need is a nexpert continually telling them it's all to hard and difficult!

Posted: **16 Mar 2021 10:41**

You may be right. It all hinges on the complexity/simplicity of the functions that Joop is envisaging. We have not got there yet.

Posted: **16 Mar 2021 11:50**

@Mike I've read the load(...) and loadfile(...) definitions and get the impression that they may be too complex for the (what I regard) simple things I have in mind. If the module solution of Helen can do the job it looks easier. But, better two options then none at all.

@Helen The idea of using a module/function to make plugin to plugin possible seems attractive but how would this work in practice? Suppose I write plugin A, you as a user download and run A and A asks you for the name of your plugin B (say 'B'). Probably B is in the public project folder? How does A run B and also important does B have acces to relevant data structures and functions in A?

Two examples:
1. Converting generic sources to templated ones.
Part of of a script for doing this is generic (A). Like getting the data from all the generic source fields, the template fields, the repositories etc. and making them available as an easy accessible structured table. Also, when the user has mapped the data in this table to the fields of the templated source (how, why etc. may fully depend on the user, country, language, customs etc.) using her/his plugin (B) plugin A takes over and creates the new templated source etc.

This plugin B is in essence rather straight forward especially when good examples are made available and I expect manageable by any average+ user.

2. Copying text data from websites in different countries/languages/formats.
This example assumes that plugin A is run as a DEA and that the data on the website in question is copied to the clipboard. Plugin A could now fetch the data from the clipboard, do some cleaning and provide it in a table to plugin B1 (this example uses 2 plugins B).
B1 now converts the website data to a structured table like e.g. fieldname, field value. From here it could go on and map the field to the source etc. (see previous example), but I think that the problem of analyzing website data is more complex that mapping a well structured table to source fields. Hence I think that plugin B2 could do the mapping. My experience is that B1 can support a number (class) of different websites and with some examples would give the average++ user a good starting point. Plugin B2 is like B in the previous example.

The next step in the DEA - mapping the source data to fact fields - usually involves complex coding (back to plugin A, that now has a well defined table to start from).

Note: the language pack approach of CP is a bit like what I've described above (as a Dutch user I did the Dutch translation). In essence it creates a number of API's for the user to do their own language specific things. I think that it's a great way of hiding complexity where possible and yet recognizing the need for supporting easily adaptable local differences.

Posted: **16 Mar 2021 12:15**

Maybe consider standing those concepts on their head.
Provide your Plugin A functions as a library module that any user can incorporate into their Plugin B using 'require'.
The model for such an approach is all the existing library modules.
Your documentation would describe the operation of each function as it would for your original approach.
You could provide an example Plugin B for some existing well-understood case as a guide to the user.

There is even a mechanism for installing library modules. See the Install Library Modules published Plugin.
I tried to do something similar for the library modules I use in my Plugins, so others could utilise them, but was considered unworkable. There are problems with maintaining backwards compatibility, which may also apply to your original Plugin A concept. What happens if you need to amend, improve, correct some functionality in Plugin A that might impact Plugin B?

Posted: **16 Mar 2021 12:55**

JoopvB wrote: ↑
16 Mar 2021 11:50
@Helen The idea of using a module/function to make plugin to plugin possible seems attractive but how would this work in practice? Suppose I write plugin A, you as a user download and run A and A asks you for the name of your plugin B (say 'B'). Probably B is in the public project folder? How does A run B and also important does B have acces to relevant data structures and functions in A?

My preliminary thoughts:

You provide a simple template plugin B/B1/B2 for a user to customise -- downloadable from the plugin store (well documented so people understand it's associated with another plugin and does nothing on its own).
A user downloads and installs the template in their plugins directory, and then edits it using the built in Lua editor. (It needs to be in the plugin directory for the built in editor to find it and I'm assuming we don't want to complicate matters for the users by requiring them to use a different editor). You will probably recommend saving it with a different name -- and establish a naming convention so that you can identify the plugins 'helpers' associated with plugin A.
You run plugin A and ask the user to select from a list of possible plugin Bs (Identified via your naming convention); they select one and you require it within plugin A, at which point you have the public functions and public data within B available to call.
If B needs access to A's data, you can pass it in as a parameter (table) to the functions in B that you call.
If B needs access to A's functions, that's a bit more complicated -- I have a way of working where A would pass a function name to B ('caller') that contains other functions and B would call (e.g.) caller.dosomethinguseful at the appropriate point (determined by your plugin).

tatewise wrote: ↑
16 Mar 2021 12:15
Maybe consider standing those concepts on their head.
Provide your Plugin A functions as a library module that any user can incorporate into their Plugin B using 'require'.
The model for such an approach is all the existing library modules.
Your documentation would describe the operation of each function as it would for your original approach.
You could provide an example Plugin B for some existing well-understood case as a guide to the user.

Doesn't that just move the complexity of coding onto the user which is what Joop is trying to avoid?

Posted: **16 Mar 2021 13:17**

Conceptually, I believe my suggestion is effectively very similar to Helen's.
It requires the user to write essentially the same Plugin B script whichever approach is adopted.
They both need much the same documentation to describe the API functions.

However, I think my approach means the user can fully syntax check their script against the library module functions, etc.
Also, there are no problems accessing functions, table structures, etc.

A variant approach avoids a formally installed library module. The Plugin Store holds a Plugin A 'template' with all the necessary functions, etc, and perhaps some sample Plugin B script. The user downloads that Plugin A, renames it, and modifies the sample Plugin B script to satisfy their needs following the documented advice.

Posted: **16 Mar 2021 14:15**

@Mike I had al look at the Install Library Modules published Plugin.
It says: This plugin will not be updated for and will not run in FH V7 as the library modules are now installed by FH. Do I understand it right that under FH7 it's not possible (anymore) to create your own set of 'utility' functions and make them available as a module to be used by anybody (through require)?

About inverting the concept of A calling B and B calling (functions of) A. I thought of that too, but considered it impossible because that would mean that A should be published as a module in the store an required by B. Because I read somewhere that is not allowed (see above) I resorted to the A calling B approach. If it would be allowed, I guess it's the most simple solution. It would be just like fh.Utils and a strict interface definition would resolve any optimization or back/forward compatibility problem.

As to your variant, do you mean that the user downloads A and plugs in his B function, at the "interface" point, changes the name of B to 'B' and run A?
This could certainly work, but it would stil expose the user to the code of A, which is what I was trying to prevent (for all sorts of obvious programmer kind of reasons

).

@Helen Walking through your thoughts:
1. Yes.
2. Yes.
3. Yes, or just provide the name and find it.
4. Yes.
5. Suppose B needs a function in A to split a name (string x) into parts according to language conventions (XX). The basic call would be something like splitName(x, XX). Here splitName is a function in A. To be able to do that, A must provide B with a table of all (API) functions of A (and document how to call them). That would - I hope - allow B to call splitName in A?

If the last point can be accomplished, I think this might be a workable scenario?

Posted: **16 Mar 2021 14:30**

JoopvB wrote: ↑
16 Mar 2021 14:15
@Mike I had al look at the Install Library Modules published Plugin.
It says: This plugin will not be updated for and will not run in FH V7 as the library modules are now installed by FH. Do I understand it right that under FH7 it's not possible (anymore) to create your own set of 'utility' functions and make them available as a module to be used by anybody (through require)?

You couldn't do it under version 6, Joop, because Install Library Modules only installs libraries hosted by CP. (As an aside, I have asked if they're open to the idea of libraries in the plugin store -- the main issue may be confusing users who aren't developers, and also for developers, selecting from overlapping solutions (Mike and I for example have overlapping libraries. Your requirement -- for libraries specific to a single plugin -- may be more palatable).

Re point 5, A doesn't need to provide B with a list of all A's functions, just the ones B is expected/allowed to use -- by something as simple as passing a table of function names into B.

Posted: **16 Mar 2021 14:55**

The only reason Install Library Modules is restricted is that it is superfluous in FH V7 as the program installation already installs the libraries in the C:\ProgramFiles(x86) folder.
It is perfectly feasible to create library modules in FH V7 and download them to the Plugins folder just like FH V6.
As Helen says there are issues with CP approving that approach and also potential library name clashes, etc.

The backward/forward compatibility issue is quite subtle. If you updated your library modules and a user installed them they would overwrite the previous version. If the user had a Plugin that relied on the earlier version it might not work with the later version that now exists in the Plugins folder. It would take a lot of careful coding or adding later versions as new functions to ensure a new library did not upset any user Plugins. It was this backwards compatibility that worried CP most when I proposed my library modules.

Unless you compile the library modules into DLL the code will always be exposed to the user.
If it is not directly in the Plugin, it is in a file in the Plugins folder that is almost as easily accessible.
You just have to impress upon users that they change your supplied code at their own risk.
It can actually be a benefit as experienced users can adapt your scripts to suit their own purposes.

A variant of the library module approach is to provide a FHUG KB Code Snippet.
An example is the User Interface > Progress Bar which is offered in two versions and I have adapted it into a library module that could easily be a Code Snippet.
The Code Snippet has all the descriptive documentation and examples of use.
The user simply downloads that script and embeds it in their own Plugin.

Posted: **16 Mar 2021 15:11**

To conclude (and check before walking down a dead end street

), I could start testing the following scenario:
1. Create a plugin A with some data and functions. This A would potentially be in the store.
2. Create a plugin B (name 'B'). This B would usually be in the users project.
3. Somewhere in the code of A, it will request the name of B.
4. It will call 'B' with a table that contains some variables and functions in A.
5. 'B' will call the functions in A (and use the data in A) and print the result.
6. 'B' returns (B-data) to A.
7. A prints B-data.

Does this approach proof that plugin A can call B, with A exposing (through an interface) data and functions in A to be used in B and B can return data to A?

Posted: **16 Mar 2021 15:20**

Mike, are you talking about plugin A (provided by Joop) calling library B (customised by user) or plugin B written by User calling library A written by Joop? A few comments assuming option 1 (which allows the user to avoid the complexity of plugin A code even if it's potentially visible to them if they go looking) :

1. As long as Joop keeps the APIs fixed, or only introduces new optional API elements, the compatibility issue becomes more manageable.
2. If a library is compiled, where might it be hosted? Not the plugin store. Not the KB. ????

tatewise wrote: ↑
16 Mar 2021 14:55
The only reason Install Library Modules is restricted is that it is superfluous in FH V7 as the program installation already installs the libraries in the C:\ProgramFiles(x86) folder.
It is perfectly feasible to create library modules in FH V7 and download them to the Plugins folder just like FH V6.

Are you offering to add them to the plugin on demand, Mike? It would make you somewhat of a bottleneck.

A snippet based approach comes back to 'where does the complexity end up?' If we want (as I assume we all do) to remove as much complexity from Lua novices as possible, asking them to knot together scary snippets is not the way to do it.

Posted: **16 Mar 2021 15:24**

JoopvB wrote: ↑
16 Mar 2021 15:11
Does this approach proof that plugin A can call B, with A exposing (through an interface) data and functions in A to be used in B and B can return data to A?

Yes. I know it works but you'll want to check for yourself.

Posted: **16 Mar 2021 15:48**

We are discussing various options with different pros and cons for Joop to choose from.

There is no problem with a Plugin in the Plugin Store being able to download library DLL files to the Plugins folder.
That is what Install Library Modules does. Those modules comprise folders, DLL, LUA, etc.
They are downloaded as a ZIP file and unpacked by the Plugin ~ hey presto! The technique works for ANY file types.
The Install Library Modules includes the script and all the tips necessary to perform a similar exercise for any other complex set of files. It is only about 100 lines of fairly simple script. I would not be a bottleneck.

Posted: **16 Mar 2021 15:58**

Joop, I've thought of another approach which should overcome most/all of the problems, but need a while to work it through... Watch this space.

Posted: **16 Mar 2021 17:07**

OK, brief outline:

One way to get around the problems of where ‘companion libraries’ (as I’ve started thinking of them) can be hosted, is not to host them at all.

Instead, have a button somewhere on the UI for the main plugin (A) that offers to create the template for a companion library (B) and when operated:

• Create a template in the fh plugins folder 'C:\ProgramData\Calico Pie\Family Historian\Plugins\x.fh_lua', named as specified by the user (basically write a fixed string to a text file – I do the same with a plugin that needs to create customised queries and reports on the fly).
• Records in a file in the plugin data folder the name of the template(s) created, so that they can be offered up for selection when A needs to require them

This approach:

1. Avoids the need for hosting/downloading the templates, keeping everything under the control of the author of plugin A (except for the hosting of plugin A) and keeping all the code you need to maintain in a single place
a. It also sidesteps the question on CP hosting libraries by plugin authors which they have declined to do in the past because of compatibility issues and the potential for conflicts between plugin authors when a user reports a problem
2. Still allows the user to customise B using the FH built in editor.
3. Ensures that any new template B is compatible with the version of the plugin A in use
a. You could include a version number in B to match the version number in A and run a compatibility check when B is selected, with a warning if there are known incompatibilities that need to be fixed (which should be rare if the APIs are fixed).

Posted: **16 Mar 2021 18:16**

Companion Libraries... a novel and interesting concept.

Let's see if I understand it from a users perspective.
1. The user downloads plugin A from the store and 'enriches' it by specifying that e.g. API 1 of plugin A should call the users own plugin B1 and API 2 of A should call B2.
2. The user can name and save this setup for later use as her/his customized version of plugin A.
3. Plugin A will, when executed, check if a record for this setup exists and, if so call B1 and B2 when appropriate.
4. If the setup is not found go to point 1.

Helen, is this what you have in mind?
It seems that something separate is needed to manage the different setups (i.e. the library).

Posted: **17 Mar 2021 09:29**

From a users perspective, I was seeing it like this.

The user downloads and installs plugin A.
Plugin A offer the user the option to create a 'companion library' B1. More than one class of companion library can be supported if necessary, but I'll concentrate on a single class for the moment as that will be simplest for the user, I suspect, if it's possible).
If no suitable companion library exists and the user chooses not to create one, A exits.
If the user decides to create a library, A creates a template version of B1 in a the fh plugins directory with the name chosen by the user (modified to show that it is associated with A so that it can be seen as such in the plugin list displayed by FH), and 'registers it' in its plugin data (save B1s name and which API it satisfies if there are more than one), then exits so that the user can edit B1 to their needs. (They can't do that while plugin A is running).
User edits B1 and saves it.
User runs A again. A detects that at least one companion library exists and offers a list of the registered libraries for selection at runtime. (A will need to check that all registered libraries exist i.e they haven't been deleted and do some error handling round missing ones)
User selects a companion library and A runs to completion.

If multiple companion libraries are needed to satisfy multiple APIs, saving previously-selected combinations makes sense for ease of use.

Posted: **17 Mar 2021 11:56**

Helen, following your list:
1. Yes.
1.1. How are we going to provide the user with dummy (example) versions of B1 and B2?
1.2 One way I can think of is extract them from A where they can be implemented to create a full functioning (default) A.
1.3 This assumes a kind of Companion Library (CL) management plugin to do the extraction.
1.4 If we assume a CL management plugin exists life gets easier. We could skip your point 2 and 3 and go straight to 4 and use the CL management plugin (CL-man) to handle that too.

2. Suppose there is B1 and B2. Since they are both 'subroutines' of A, I expect they both belong in Companion Library (CL) B? And if their is going to be a plugin AA with BB1 and BB2 that would belong to CL BB?
3. Yes; same in case of CL-man.
4. Yes; in case of CL-man this would all be handled by this plugin.
5. Yes.
6. Yes; however, in case of CL-man doing the selection, A could run without asking anything about CL. From the users point of view, 'his/her' A would have been built from 'Lego' pieces A, B1 and B2.
7. Yes.

What are your ideas about CL-man? Doable, sensible or too much or ... (to make it an 'open' multiple choice

).

I'll make a prototype A, B1 and B2 and try it out (may take a few days, family priorities sometime interfere...

).

Posted: **17 Mar 2021 15:02**

I think your introduction of a CL management plugin is over-complicating things. We are after all only talking about text files.

You can provide a dummy example of each of B1 and B2 in the text of the plugin help on the plugin store, behind an accordion or in a tab if you want to avoid cluttering the main help. (Although the help has been removed from the search because there was so much of it that it was making it very difficult to find plugins -- there's an index page instead, so multiple pages will work without causing problems).

If you create a barebones full functioning default B then you can include that as a string inside A and write it to a text file on demand. A will need to keep track of its companions anyway.

Your point 6: it depends how static the selection of companion is -- if there's only ever one version of a companion per user, A can use the first one created and not ask for a selection or offer to create more. If the user might want different behaviour at different times (e.g. different handling for different source types) and you want to keep the companions very simple, you'd probably want the selection interface in A.

Family Historian User Group

Can a plugin call a plugin?

Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?

Re: Can a plugin call a plugin?