* Restoring Settings to New OS Build

[tatewise]

Can't you read the format from the registry and then use that to parse the modified file date to work out which parts of the string the day is, which part the year is, etc. and then once you have the date parts, create a standard string for comparison?

Yes, that might be possible, but there are many formats and without going through the locale settings for every country may still not cater for parsing all of them. The UK and USA alone have about ten formats.

[tatewise]

Perhaps one more step if you don’t do it already? An important part of risk management in general is controlling and minimising exposure to that risk. If a plugin used standard library functions where possible, and the riskier Registry hack only where necessary, that would eliminate the remaining risk completely for probably >99% of real world FH applications that use Latin filenames.

I know I’m always banging on about keeping code as simple as possible, but I think the slight extra complexity is justified here.

Yes, I imagine the plugin could use the Lua lfs modification attribute for files with ANSI file paths and only use the Registry hack otherwise. The plugin is comparing the modification date of two files (the master settings fie and the backup copy) so if either has a Unicode UTF file path then both must use the same technique to ensure they are comparing apples with apples, which makes the code even more complex.

I am anticipating the number of real world FH users with Unicode UTF file paths will be growing now that FH supports many foreign language packs as well as all Unicode characters. That is why my plugins are switching over to my variant of the fhFileUtils library using File System Objects to support Unicode UTF file paths in all FH versions. The downside is issues with such as file modification dates and other Lua library related features.

[NickWalker]

But if you look at the sShortDate and sShortTime you only need to identify where the d (or dd) is, where the M or MM is, where the yyyy is, etc. and then you can interpret any file modified date. So there may be many different formats but for 99.9% of FH users they will include those elements. You could build in some checks so that if you don't find any of those elements in the ShortDate you could say "This plugin is unable to deal with backups due to the date and time formats on this computer".

[tatewise]

Nick, that was my first thought but there are further complications.
Some formats have MMM for the month in letters, so that needs month name conversion tables for appropriate languages.
There are several different delimiters between the d, M, y items such as / : . that can include foreign language symbols such as Japanese for Day, Month & Year.

The time formats also have a variety of delimiters and may be suffixed by am, pm, AM or PM for 12 hour times.

I am simply amazed that there is no process that I can find to extract and convert file modified or created date-time values into any standard format or the Unix epoch seconds since 1-1-1970 as provided by lfs. Believe me, I have searched high and low.

I'm not sure whether a user would prefer the plugin to abort dies to date formats or run the miniscule risk of the format being changed accidentally.

[NickWalker]

Yes I realised that. I think that things like MMM could be categorised into the "unsupported" category. PM could just be ignored and add 12 onto the hours, etc. It's definitely not straightforward I agree, but I think it is really not acceptable to be changing anyone's windows date/time format in the registry. Another process running at the same time as your plugin (e.g. anti-virus, backup software, application software, utilities, etc.) could be accessing that same data at the time and who knows what effects it might have (probably none but you never know).

I assume the issue is with the method you're using to access the FileSystemObject? If I was accessing that in a .Net language the type returned from the function would be a date type from which I could extract the information I need. Correct me if I'm wrong, but it appears the function you're calling is returning the date modified as a string and in the format set by the user which is hopelessly inadequate. Are there any other Lua libraries you could use to get the file modified time? Could CP be persuaded to add some functions like this and some date conversion functions too?

[Mark1834]

...now that FH supports many foreign language packs as well as all Unicode characters. That is why my plugins are switching over to my variant of the fhFileUtils library using File System Objects to support Unicode UTF file paths in all FH versions.

I'm not sure that's a consistent argument, as language packs are only applicable to FH7, and the FH5/6 user base will continue to dwindle over time.

Personally, I think going down your own parochial path like this is not the way forward. As for added complexity to eliminate unnecessary risk, it's probably half a dozen lines - not really significant in a plugin that takes over 7000 lines to copy two folders and two Registry keys...

[ADC65]

I have been very grateful for this plugin, but I would be unhappy to use it if it made unwarranted (in my view) changes to Windows System settings that had no direct relevance to the task I am carrying out.

At the very least I think you would need to put a very prominent warning at the start of the plugin, and this would surely put a lot of people off using it.

I don't underestimate the complexity of the task, and unfortunately I don't have the technical know-how in this area to suggest any solutions.

I'm not sure why the plugin is looking at file dates in the first place? My backup folders are about 20Mb in size - trivial by today's standards. Would a simpler plugin be justified that just creates backups and restores them on request without the extra bells and whistles? Something along the lines that Mark1834 has suggested in a batch file? I'm in no way denigrating the current code, but maybe it's getting too complicated?

[Mark1834]

I think it is really not acceptable to be changing anyone's windows date/time format in the registry

I agree, but the key opinion is that of CP. If their plugin police deem that unacceptable behaviour for a store plugin, it ain't going to happen. Has anybody checked?

[ColeValleyGirl]

Are there any other Lua libraries you could use to get the file modified time?

lfs (LuaFileSystem) only handles filenames in the local code page, and we know that cause problems for some existing users (whose language doesn't fit in a single code page!). Penlight relies on lfs so has the same problem.

Luacom allows use of Microsoft com objects, so maybe there's a com object that would help.

I've asked on StackOverflow in case there's any bright idea we're missing.

[ColeValleyGirl]

I think it is really not acceptable to be changing anyone's windows date/time format in the registry

I agree, but the key opinion is that of CP. If their plugin police deem that unacceptable behaviour for a store plugin, it ain't going to happen. Has anybody checked?

I rather suspect the admins wouldn't allow it through...

[tatewise]

Nick, the only option in the Microsoft FileSystemObject library is the DateLastModified property that applies the current locale format to the returned date-time text string.

Mark, I don't understand your point about "the FH5/6 user base will continue to dwindle" as that means the probability of file paths with Unicode UTF characters is only going to increase and that is what fhFileUtils is meant to support as a replacement for Lua libraries that only support ANSI file paths. The fact that my variant also supports FH V5/6 is not significant to the discussion.

Having said all that, I needed a quick solution to the recent plugin problem that users (in USA) were experiencing.
Now that the crisis seems to be over, I have more time to consider all the possible alternative solutions.
I have checked Microsoft libraries and reviewed similar request on StackOverflow without success.

I even considered applying the date text string to an FH API Date object and using the FH API Date compare before comparing the time text strings, but suspected that would not work as it is impossible to differentiate UK dd/mm/yyyy formats from USA mm/dd/yyyy formats and other formats may be unsupported by FH.

The updated plugin with the Registry hack is published in the Plugin Store, so was approved by CP.

Adrian, the plugin compares the date-times of files to prevent old Backup files from overwriting later ProgramData files for the same custom item. I'm sure you would not want your latest version of a Query or Fact Set to revert to an earlier version when restoring from backups.

[ColeValleyGirl]

Mike, haven't had time to experiment but...

https://docs.microsoft.com/en-us/window ... emdatetime

[tatewise]

Yes, I saw SWbemDateTime and tried to get it working in a plugin without success but I needed a quick fix
There needs to be a way to feed the FSO.DateLastModified text string into the SWbemDateTime.Value then it should provide the properties for year, month, day, hour, minutes, seconds that can be compared with another value.

[Mark1834]

Adrian, the plugin compares the date-times of files to prevent old Backup files from overwriting later ProgramData files for the same custom item. I'm sure you would not want your latest version of a Query or Fact Set to revert to an earlier version when restoring from backups.

That's the key advantage of the Windows approach for this particular application. The simple 4-line templates are just that - templates that do an unconditional one way copy that users can take and modify for their preferred approach. Avoiding overwriting later files would be a simple change, and I suspect many more users would be capable of modifying a Windows script than a long complicated Lua plugin...

It's a case of using the appropriate tools for the job in hand. Backup and restore is a Windows activity, so do it in Windows so you don't have to worry about any of these issues around languages and messing with the Registry. Keep plugins for their intended purpose - processing and modifying FH data.

My point around the dwindling FH5/6 user base is that all new multi-language users will be FH7, and having multiple solutions for the same problem doesn't feel healthy. IMO, it's better to try and fix the shortcomings of the supplied tools, rather than develop your own rival forks.

[tatewise]

OK, so publish the simple scripts in the KB with suitable instructions and give users the option.
I would be more than happy to scrap my plugin. It is becoming a pain to maintain. If only CP had its own Backup & Restore.
If there was a fix available for the supplied tools then I would use it, and if I find a workaround I will try and get it incorporated into a supplied tool.
The main variation in my fork is the suggestion you made yourself to cope with Unicode UTF filepaths by copying to a 'safe' filepath, and that variation is absolutely needed for FH V7 whilst also working for FH V5/6. I am also aware that the fhFileUtils library is under review with changes that I understand to be similar to some of my variations. Essentially, my variation is a copy of the fhFileUtils code embedded rather than required plus a few extra workarounds for invoking Lua libraries that demand ANSI filepaths.

[ADC65]

Adrian, the plugin compares the date-times of files to prevent old Backup files from overwriting later ProgramData files for the same custom item. I'm sure you would not want your latest version of a Query or Fact Set to revert to an earlier version when restoring from backups.

Thanks for the clarification Mike, I thought it would be something along these lines. You're right when you say I wouldn't want later version overwritten, but for me personally this would not be a problem as I do very regular backups and keep the resulting date-stamped folders for at least a year. My biggest problem, not that it's happened yet, would be picking the appropriate backup file to restore! Although this would nearly always be the last backup for me. I understand different people work in different ways and this will not work for everyone.

I have tried out Mark1834's script and it works for me; I can understand what it's doing and I'm computer-savvy enough to change what I need in it. I'm sad to say I won't be updating the plugin to the version with a registry hack in it, I'm just not comfortable with that, but I wish you good luck trying to find other solutions. You work very hard in producing work that helps others enormously, and I thank you for it.

[ADC65]

If only CP had its own Backup & Restore.

Hear, hear! Is there a wish list item for this we can vote on?

[tatewise]

There is a related Wish List Ref 460 Localise Settings to Projects that proposes an alternative solution with significant comments.
I am sure CP are aware of the issue but the motivation to implement anything is diluted by the existing workarounds.

[Mark1834]

The Wish list does have exactly that, but it is officially marked as solved by the plugin. That’s the paradox noted earlier - plugin fixes take the item off CP’s radar and pass the responsibility for future maintenance to the plugin author (or other users, as plugins are open source community products).

The plugin has served us well over many years. I’m sure we are all grateful to Mike for the time he has spent on it over the years, and it still has a place in our toolbox for users who prefer a pre-configured “out of the box” solution.

[rob p]

The related Wish item Ref 460 Localise Settings to Projects would help resolve this issue, as well as another nagging problem I’ve had with FH since I came here (only a year ago). There are some settings I switch back-and-forth for different projects – It’s become “muscle-memory”, but it would be sweet to not have to do that.

I understand that CP’s priorities are “diluted” by “successful” plugins.

Is this Wish List item truly classified as “solved by plugin”? If it were implemented, wouldn’t the FH-native Backup & Restore do what we need here?

[BillH]

That's the key advantage of the Windows approach for this particular application. The simple 4-line templates are just that - templates that do an unconditional one way copy that users can take and modify for their preferred approach. Avoiding overwriting later files would be a simple change, and I suspect many more users would be capable of modifying a Windows script than a long complicated Lua plugin...

While I agree there would probably be more people able to modify a batch file than a plugin, I for one would have absolutely no idea how to change the batch file to not overwrite newer files for example. I wouldn't know a batch file from a batch of cookies.

That is why for me the plugin is so great (please Mike don't quit maintaining it). I agree that if you can determine a way to make it work without the registry hack, that would probably be better. I for one would be willing to take the risk of using it it. The advantages of the plugin seem to outweigh the risk for me.

Bill

[Mark1834]

I'm working on an idea that might give us the simplicity and extra functionality of doing it in Windows and the ease of use and flexibility of the plugin...

[tatewise]

The old Wish List entry is Ref 118 Backup and or archive custom reports, queries and text schemes.
Strangely, it says it was Added/Released in Version 3.1 of FH
That should be FH Version 5.0 when plugins were introduced.
I have no idea where 3.1 comes from as in 2012 when the plugin was written it would have been 1.0.

FYI: The plugin has been removed from the Plugin Store by CP as they have just realised the Plugin for Backup and Restore Settings is writing to a registry key outside of the Family Historian settings.
The Plugin will remain unavailable until a suitable workaround can be designed.
So the sooner you can offer your solution Mark the better!

[ColeValleyGirl]

Mike, can you not just revert to a previously released version in the store without the registry manipulation, with a caveat about date formats in the Help?

Another option might be to use lfs for the file handling, but trap any errors when it can't access a file and use the fso mechanism to get the (text) dates and prompt the user which they want to keep? It should only apply to a minority of files at present (although it's not a long term solution).

[ColeValleyGirl]

Mike does this help (from the luacom documentation at http://webserver2.tecgraf.puc-rio.br/~r ... ode45.html)

3.4.6 DATE type
When converting from COM to Lua, the default behavior is to transform DATE values to strings formatted according to the current locale. The converse is true: LuaCOM converts strings formatted according to the current locale to DATE values. The script can change the conversion from strings to tables by setting the DateFormat field of the luacom table (the LuaCOM namespace) to the string "table". The table will have Day, DayOfWeek, Month, Year, Hour, Minute, Second, and Milliseconds fields. To return the conversion to strings, set the DateFormat field to "string". Be careful with this feature, as it may break compatibility with other scripts.