* Restoring Settings to New OS Build

[deckie49]

Greetings,
I've run into an issue restoring FH settings onto a new install using Win 10 Pro v. 21H2.
I upgraded my OS from 7. Download and installation of FH seemed to go smooth. However, using the "Backup and Restore Plugin v3.2"
I get the following error. The plugin freezes from there.
I have no clue how to proceed. Can someone kindly help?
Thankk you!

[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4799: attempt to compare nil with number
stack traceback:
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4799: in upvalue 'getTime'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4815: in upvalue 'intTime'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5967: in function <[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5898>
(...tail calls...)
[C]: in function 'iuplua.MainLoop'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."] in field 'ShowDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:6257: in function 'GUI_MainDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:6297: in main chunk

[tatewise]

Sorry about that. I have moved this to the Plugin Discussions forum where the same problem has been reported and a fix is being checked in the posting Problem setting up new laptop (20493).

Does the plugin attached there also fix your problem?

[Mark1834]

Given these issues, is it time to consider an alternative method for this scenario?

Copying settings from an existing FH installation to a brand new one is simply a matter of copying two folders and two Registry keys. It doesn’t need lots of options and fine control, and is purely a Windows operation. Therefore, do it in Windows, without the complexity of a Windows app running a Lua script that calls a Lua library that calls a Windows object that completes the circle by interacting directly with the Windows API...

That’s a lot of elements that have to play together, and a simple error such as US vs EU date format causes it to fail.

One 4-line script to backup, and another 4 lines to restore, rather than thousands of lines of plugin code. Job done. Even if you add things like checking for 32-bit vs 64-bit, it’s still a simple script that’s easy to maintain and easy to use (and doesn’t have to jump through extra hoops restoring the Registry from within FH).

Can we build on that to develop a simple and reliable method that we can all recommend?

[ADC65]

Would it be possible to encourage Calico Pie to incorporate a proper settings backup in their product? I find it crazy that such a highly configurable piece of software relies on the know-how and generosity of users to carry out this task. Before I came across the Backup & Restore plugin (or possibly before it was written) it was a major inconvenience to restore all the settings and tweaks after a computer rebuild.

I know this isn't quite the question you asked. I would be happy to work on or test any user solution. But I think it's quite a gap in the program itself.

[Woodg]

Would it be possible to encourage Calico Pie to incorporate a proper settings backup in their product? I find it crazy that such a highly configurable piece of software relies on the know-how and generosity of users to carry out this task.

I know this isn't quite the question you asked. I would be happy to work on or test any user solution. But I think it's quite a gap in the program itself.

My thoughts exactly. For me, I delayed selecting FH for a couple of years as my new software solution because of this issue. My thoughts, at the time, was that if the developer couldn't create a proper backup and restore solution then what other shortcomings might FH have. It took me a while to accept that much of the usefulness of FH was its capability of being enhanced by plugins created by third parties. Nonetheless Backup and Restore should be part of the core product.

(BTW, a big thank you to those who create the plugins.)

Glenn

[Mark1834]

I agree fully. It is a glaring omission in FH, but unfortunately we are in a Catch-22. CP are a tiny outfit with limited resources. What is the right balance between fixing holes and providing new features? Plugins are a great feature, but in my view should be used mainly for niche operations that might apply to only a limited number of users. While there are users who are prepared to spend a huge amount of time fixing the holes (and we are all grateful to them), is it best use of CP's time to duplicate that work at the expense of other activities? That's the paradox that has been there for years, and probably won't get resolved anytime soon.

On the specifics of this issue, the following Windows script backs up all your settings to a specified backup folder, and the second script restores them back again by simply reversing the commands. It's a blunt "copy everything", but fine control isn't needed for setting up a new installation. It restores the same default project and backup folders as your original installation, but these can be simply changed in FH the first time you run it if necessary.

Code: Select all

robocopy "%PROGRAMDATA%\Calico Pie\Family Historian" X:\fhBackup\fhProgramData /MIR
robocopy "%APPDATA%\Calico Pie\Family Historian" X:\fhBackup\fhAppData /MIR
Reg export "HKEY_CURRENT_USER\SOFTWARE\Calico Pie\Family Historian" X:\fhBackup\fhCURRENT_USER.reg /y
Reg export "HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Calico Pie\Family Historian\2.0\Preferences" X:\fhBackup\fhLOCAL_MACHINE.reg /y

Code: Select all

robocopy X:\fhBackup\fhProgramData "%PROGRAMDATA%\Calico Pie\Family Historian" /MIR
robocopy X:\fhBackup\fhAppData "%APPDATA%\Calico Pie\Family Historian" /MIR
Reg import X:\fhBackup\fhCURRENT_USER.reg
Reg import X:\fhBackup\fhLOCAL_MACHINE.reg

As written, it assumes 64-bit Windows on both PCs (virtually all are today). One of the Registry key paths would need editing for 32-bit.

Users are welcome to play with it. It's a prototype template, so only use it if you understand the file structure and are happy creating, modifying and running batch files. I actually use a slight variation for my routine backup, where I call 7-zip to create single file zip archives of the folders, rather than thousands of individual files.

With this approach, setting up a new installation follows these steps (but remember it must be the same main version of FH on both PCs, and the target PC revision should not be older than that on the source PC (unlikely, as the latest version is readily available).

1. Run the backup script, either from the command line or by double clicking on the batch file. FH should be closed, to ensure that there are no unsaved changes in settings.
2. Install the new copy of FH, allow it to activate, and then close it.
3. Run the restore script. All your settings are now identical in both installations.

Copying is reliable and robust, as it is all done within Windows, but this is a prototype - please find the potential problems, and by all means suggest improvements and refinements. It worked flawlessly in my recent installation, but I want you to find the gotchas that I've not thought of...

[tatewise]

I fully agree that FH should have had a comprehensive Backup & Restore settings feature from the outset.
A few settings are migrating to the Project folder, so are included in Project Backups.

Mark, when you rewrote the Backup and Recovery advice recently, I'd fully expected the script you posted here to be included. It certainly offers a simpler solution for many (but not all) scenarios.

The plugin was first written many years ago, and deals with more scenarios, some of which no longer exist in FH V7, which makes the simple script a feasible alternative.
The plugin supports Tutorial files for FH V5, Diagram Icon files that can be anywhere in FH V5/6, and Diagram Picture files that can still be anywhere in FH V7 that the simple script does not include.
It also copes with customisations that may have been updated on either PC, whereas the simple script requires all customisation updates to flow in one direction.
Finally, the plugin copes with Wine/Crossover based FH that does not support Reg Export and Reg Import commands.

So the plugin is crucial for some scenarios but the simple script is a better solution for a subset of scenarios.

[Mark1834]

Thanks Mike, that’s a very fair comparison. I deliberately restricted the KB to a general description of the approach, as I felt it was a little self-indulgent to bang on too much about my preference if nobody else is using it...

Once we get a bit more feedback on successful application, it probably can be added to the KB with the indicated scope. I think specifying %OneDrive%\Documents as the default path is probably better than a defined external drive. Folks that use OneDrive can use the scripts completely unmodified, and settings backups are synced automatically. I have a Family Historian\Settings folder that sits alongside Family Historian\Projects and gets included in my routine backups.

[jbtapscott]

I have always used the Plugin for my backups but thought I would give Mark's scripts a whirl as, if successful, I could add a Backup cmd file to the Task Scheduler and automate the whole backup process (I currently rely on a weekly Outlook Calendar "reminder"!).
For the record, I modified the destination on the first line to "%USERPROFILE%\Documents\Backups\Family Historian\fhProgramData" and the remaining lines to something similar (I don't use OneDrive or anything similar) - the content of my Documents folder is backed up to the cloud multiple times each day.
After making these changes, I successfully ran the script and, at a top level, everything appears in the backup folder as expected. The only thing I noticed is that the fhProgramData folder (as created by the script) is appx 9Mb larger than the Program Data folder (as created by the Plugin) and contains 180 folders (71 by the Plugin) and 2,354 files (1,765 plugin). I haven't had a chance to investigate the differences yet, and suspect there may be a simple explanation, but I will spend a few hours over the next day or so checking through the results.

Perhaps I should also add that I have successfully used the Plugin for restore purposes on multiple occasions (new machines and also operating system "resets"), so really will not be sure everything is okay with the script approach until the next restore!

EDIT - have now compared the folders and the difference is the Map folder which the Plugin does not appear to populate (probably because it is a cache?)

[Mark1834]

Thanks Brent. That makes sense, as the map is saved as a huge number of small png files that can take ages to save to a slow drive such as a USB stick.

[tatewise]

Well spotted. I'd forgotten that 'tweak' in the plugin that omits the Map folder as it is only a non-essential cache.

Beware of long backup paths such as:
%USERPROFILE%\Documents\Backups\Family Historian\fhProgramData
that are longer than the source path:
%PROGRAMDATA%\Calico Pie\Family Historian
because that may exceed the maximum path length of 255 characters for some custom files.
e.g. This one is approaching 200 characters:
C:\Users\Michael\OneDrive\Documents\Backups\Family Historian\fhProgramData\Text Schemes\Custom\Birth, Marr, Death, Occups, Resids, Rec Notes + notes (custom).fht
The plugin takes care of such excessive path length cases.

[Mark1834]

Good point. I’ll add an optional line to do the same from the script...

[deckie49]

Sorry about that. I have moved this to the Plugin Discussions forum where the same problem has been reported and a fix is being checked in the posting Problem setting up new laptop (20493).

Does the plugin attached there also fix your problem?

Mike,
Thank you for your efforts in fixing the issue. The updated plugin gave me the same error.

[tatewise]

I'm not totally surprised because there are more Date-Time formats than I realised.
The attached Backup and Restore Family Historian Settings plugin Version 3.3.1 Date 28 Mar 22 uses a new technique that should cope with far more formats. Does it work for you?

If not, what are your Windows settings in Start > Settings > Time & Language > Region under Regional format data?
A screenshot would be useful. ( See Forum Usage Tips under Attachments. )

[Mark1834]

I’m curious - is the issue one of not being able to get file date-stamps from UTF file names in a “seconds since start of epoch” format, so having to use local text formats? The workaround I’ve used before is to copy the file to a temporary file with a “safe” name and use a standard library routine. It’s not particularly elegant, but it’s universally applicable.

[tatewise]

Yes, that is the problem, and what you suggest is a workaround that I have used elsewhere, but the plugin needs the last modified date-time for every file that is being backed-up/restored to determine which is the latest. That could involve a lot of file copying if the UTF file name is a folder in the Backup Folder path. So I was hoping for a more efficient method.
The plugin is using FileSystemObject tools similar to fhFileUtils, but they only provide file last modified date-time in the locale text format that depends on the country and may be chosen by the user.

[tatewise]

My latest 'solution' is to temporarily replace the current Short Date format in the Windows Registry key:
HKCU\\Control Panel\\International\\sShortDate with the format "yyyy-MM-dd" and should be very efficient.
Then the FileSystemObject file DateLastModified is in a known format and the year, month & day are easily extracted.
( The Long Time format is not so variable and the hour, minute & second are easily extracted. )
Those extracted values are supplied to the Lua os.time(...) function to obtain the Unix epoch date-time in seconds.
There is a minor wrinkle for Daylight Saving Time but that usually gives the same Unix value as lfs, and is at most one hour adrift, which is not a problem in this scenario.

My only concern is whether such a tactic works for very different cultures such as Chinese, Japanese, etc, where date and time may have different representations.

[Mark1834]

That's a neat trick, but what happens if the plugin crashes or the user interrupts execution prior to restoring the original value? Haven't you messed up their PC settings? IMO, that's not something a well-behaved script should risk.

[tatewise]

I understand that risk, and when I say 'temporarily' it is just for a dozen or so lines of script every time a DateLastModified needs converting. Also, that script will have failures trapped by using the pcall(...) function, and the user dialogue is disabled, so that mitigates almost all risks.

[ColeValleyGirl]

Mark, what you're commenting about is what I understand as 'thread-safety', i.e. the action affects code outside the plugin (including code outside FH which isn't disabled by a plugin) and doesn't failsafe.

[Mark1834]

I hadn’t heard that term before, but yes, that’s a reasonable description. The plugin is messing with stuff outside its remit, and while the risk is low if it is used as the author intended, a lot of software issues arise when the user does something unexpected - e.g. runs it in the debugger to see how it works, and aborts execution part way through.

I’d accept a risk to my FH installation if I’m hacking a plugin, but I’d be a bit peed off if it affected my OS as well.

[tatewise]

I've reviewed the script and now there are only two statements that run the debugging risk.
It could all be avoided if there was a way of getting DateLastModified via FSO (or any other method) in a fixed format or even better in UNIX epoch date-time form, without having to jump through hoops, copying files, or whatever.
e.g.

Code: Select all

StrShortDate  = "HKCU\\Control Panel\\International\\sShortDate"

local function strDateLastModified(strFilePath)
	iup_gui.PutRegKey(StrShortDate,"yyyy-MM-dd","REG_SZ")	 -- Set desired Short Date format
	local fileObject = general.FSO:GetFile(strFilePath)
	return fileObject.DateLastModified
end -- local function strDateLastModified

local strShortDate = iup_gui.GetRegKey(StrShortDate)		 -- Get current Short Date format
local isOK, strDateTime = pcall(strDateLastModified,strFilePath) -- Trap errors
iup_gui.PutRegKey(StrShortDate,strShortDate,"REG_SZ")		 -- Put current Short Date format back
if not isOK then error(strDateTime) end				 -- Report any error

[NickWalker]

Can't you read the format from the registry and then use that to parse the modified file date to work out which parts of the string the day is, which part the year is, etc. and then once you have the date parts, create a standard string for comparison?

[Mark1834]

Perhaps one more step if you don’t do it already? An important part of risk management in general is controlling and minimising exposure to that risk. If a plugin used standard library functions where possible, and the riskier Registry hack only where necessary, that would eliminate the remaining risk completely for probably >99% of real world FH applications that use Latin filenames.

I know I’m always banging on about keeping code as simple as possible, but I think the slight extra complexity is justified here.

[deckie49]

I'm not totally surprised because there are more Date-Time formats than I realised.
The attached Backup and Restore Family Historian Settings plugin Version 3.3.1 Date 28 Mar 22 uses a new technique that should cope with far more formats. Does it work for you?

If not, what are your Windows settings in Start > Settings > Time & Language > Region under Regional format data?
A screenshot would be useful. ( See Forum Usage Tips under Attachments. )

Mike,
Here is the screen shot you requested...

Fyi, the latest 3.3.1 Plugin version works like a charm!
Thank you!