* Backup and Restore FH Settings via Windows - new plugin

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

Backing up and restoring FH settings is simply a matter of copying the appropriate files and Registry settings. The existing backup and restore plugin manages the interaction with Windows. For those who are more experienced PC users, backing up and restoring the two relevant folders and two Registry via batch scripts can offer a more convenient and powerful alternative for integrating into their general backup strategy, but working out the detailed syntax can be complex.

The attached plugin aims to combine these approaches by creating suitable scripts automatically, and optionally also running the backup, so detailed knowledge of Windows commands is not required.
Plugin window
Plugin window
Capture.PNG (16.37 KiB) Viewed 3485 times
Click on Select to select the backup location. By default, all your settings (apart from the map cache) are copied to this location, overwriting any previous backup, but can be varied by selecting options.

If you have 7-Zip installed on your system, this can be used to compress the folder backups to single files, which can be more convenient for subsequent copying.

Folder and Registry key files can be annotated with the current date if you prefer to keep multiple backups.

Click on Script to generate a Windows batch file that implements your defined backup. Click on Backup to run the backup now.

Once you have this backup script, you can run exactly the same backup at any time, even when not using FH. Create a shortcut on your desktop for a simple one-click full backup. Add it to Task Scheduler to backup completely automatically at defined times.

Options that are not valid are greyed out. For example, if you select dated backups, that is only suitable for immediate backup within the plugin, as it writes a fixed date into the script.

There is deliberately no immediate restore option within the plugin. Restoring settings is best done while FH is not running, so there is a parallel option to create a restoration batch file corresponding to the selected options. Simply run this file by double-clicking on it after you have closed FH.

By default, the restore script does not overwrite newer files in FH or delete files that are not in the backup, so any changes made since backing up are preserved. Select “Force exact copy of backup configuration" to change this such that the restored settings are identical to those in the backup.

If you are interested in exploring this option, download the plugin and have a play. It does not change any of your FH data or settings until you actually run the restore script, and works in both FH7 and FH6.

My intention is that this method is easy to use for less experienced PC users, but also provides a framework for more experienced users to modify and adapt for their own circumstances.

Are there any significant options missing? I’ve tried to make the interface as uncluttered and simple as possible with a similar appearance and layout to other FH forms (it supports FH7 custom fonts and zoom), but is it easy to understand?

(EDIT: Attachment deleted - see updated version in later post)
Last edited by Mark1834 on 07 Apr 2022 13:49, edited 1 time in total.
Mark Draper
User avatar
mjashby
Megastar
Posts: 719
Joined: 23 Oct 2004 10:45
Family Historian: V7
Location: Yorkshire

Re: Backup and Restore FH Settings via Windows - new plugin

Post by mjashby »

Mark,

One quick question (prior to playing):

If the aim of the plugin/script is to employ existing Windows capabilities, why add a 7zip option (requiring additional software) rather than simply using Windows inbuilt ability to compact/extract data to/from zip files?

Mervyn
User avatar
ColeValleyGirl
Megastar
Posts: 5464
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by ColeValleyGirl »

I'd also like to see the ability to zip the files without extra software.
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

Probably because I didn’t realise that you could do it directly from the command line :D

That’s first item on the list for version 2.
Mark Draper
User avatar
ADC65
Superstar
Posts: 457
Joined: 09 Jul 2007 10:27
Family Historian: V7

Re: Backup and Restore FH Settings via Windows - new plugin

Post by ADC65 »

The Backup seemed to work fine for me.

The Restore failed with the following messages. I wasn't 100% certain what I needed to do. The plugin created a script in the Backup folder and I just ran that.
Screenshot 2022-04-07 140238.jpg
Screenshot 2022-04-07 140238.jpg (77.68 KiB) Viewed 3455 times
This is the batch file that was created:
Screenshot 2022-04-07 140340.jpg
Screenshot 2022-04-07 140340.jpg (39.41 KiB) Viewed 3455 times
Adrian Cook
Researching Cook, Summers, Phipps and Bradford, mainly in Wales and the South West of England
User avatar
Ron Melby
Megastar
Posts: 917
Joined: 15 Nov 2016 15:40
Family Historian: V6.2

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Ron Melby »

the issue is there is a space before /MIR likely a typo.
FH V.6.2.7 Win 10 64 bit
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

Sorry about that Adrian - there should be a space between the two double quotes in the first two lines of the restore script. If you add that in, it should work ok. I'll fix that straight away as it stops the restore from working without a manual edit. Zips can wait for V3!
Mark Draper
User avatar
ADC65
Superstar
Posts: 457
Joined: 09 Jul 2007 10:27
Family Historian: V7

Re: Backup and Restore FH Settings via Windows - new plugin

Post by ADC65 »

Thanks Mark, I added a space on both lines and it worked fine. I will check in detail later but so far everything looks like it should. Thanks for writing this.
Adrian Cook
Researching Cook, Summers, Phipps and Bradford, mainly in Wales and the South West of England
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

This quick update corrects the restore script typo.

Edited to remove obsolete version of plugin.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by tatewise »

Mark,
If the Append today's date... option is ticked then both Script buttons are greyed out so scripts can not be created.

In this latest plugin version, in the fhRestore.bat file, the robocopy qualifiers have changed from /MIR to /E /XO
Is that intentional?

Presumably, the Help button will link to Plugin Store Help eventually.

There needs to be a caveat regarding what image files may not be included in the backups.
If any Diagrams or Charts involve image files not held within %PROGRAMDATA% or %APPDATA% then they aren't included.
In FH V6 that can include any Icon files.
In both FH V6 & V7 it can include images inserted using the Diagram > Insert into Diagram > Picture... option.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

If the Append today's date... option is ticked then both Script buttons are greyed out so scripts can not be created.
That's the scenario I described above. It appends today's date to file and folder names, but it is not a dynamic date. Every time you use the same script in the future, it uses the same date. If you want to keep multiple backups, you have to run the backup option directly from within the plugin, and that creates a new script file every time. I don't think it is difficult to create a dynamic date with the appropriate use of variables in the batch file, but it's too much detail for a prototype.
In this latest plugin version, in the fhRestore.bat file, the robocopy qualifiers have changed from /MIR to /E /XO. Is that intentional?
Yes. They implement what is selected in the menu. /MIR makes the source and destination identical, and is the way the early scripts were set up. /E /XO copies recursively without deleting files that are not in the backup (/E), and does not overwrite newer files (/XO). If you select the "Force exact copy of backup configuration", the /MIR qualifier is used instead.

I regard icons as user data, which are out of scope. That will be explained in more detail in the dedicated help page. As you have seen, the current Help button just links back to this thread for users who can't remember where they got the plugin from!
Mark Draper
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by tatewise »

If the Append today's date... option is ticked, how do I create a fhRestore.bat script to restore my dated backup?

The icons image file issue only applies to FH V6.
The Diagram Pictures image file issue applies to both FH V6 and V7 and should also be mentioned.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

Currently, you can’t (and as CP often say, “this is by design” ;)).

If you want to create a backup for immediate use, say to restore to a new PC, don’t select the dated option. This is more for users who want to keep a series of backups “just in case”, and if restoring one of these is required, the necessary script would be devised manually.

A refinement for the next version may be to allow a fhRestore.bat script to be created, but with the backup date appended to the script name (so fhRestore_2022-04-07.bat, for example). That would create a series of restore scripts to accompany the individual backups. That seems to make sense at the moment, but I’ll have a play over the next few days.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by tatewise »

When you create the fhBackup.bat file for dated backups why not create the fhRestore_yyyy-mm-dd.bat file at the same time?
The same scenario affects the 7-Zip option, i.e. no fhRestore.bat file, which would be rather trickier to engineer.

When I used the 7-Zip option it included the Family Historian\Map\... folders that I thought should be excluded!

I had expected that all the backups would be zipped into one ZIP file, i.e. ProgramData, AppData and both REG files.
Then only one file has to be copied.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

tatewise wrote: 07 Apr 2022 17:44 When you create the fhBackup.bat file for dated backups why not create the fhRestore_yyyy-mm-dd.bat file at the same time?
That's exactly what I suggested 18 minutes earlier...
A refinement for the next version may be to allow a fhRestore.bat script to be created, but with the backup date appended to the script name (so fhRestore_2022-04-07.bat, for example).
7-Zip options are limited by what the app is capable of (and my ability to comprehend the obscure and poorly documented command line syntax!). I prefer Mervyn & Helen's suggestion of doing zipping within Windows, so that's the route I'll be exploring for future enhancements.
Mark Draper
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by tatewise »

Mark1834 wrote: 07 Apr 2022 18:31 That's exactly what I suggested 18 minutes earlier...
I wasn't sure if that suggestion was for a separate user-initiated option or automatically when Backup performed.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by tatewise »

A couple of further thoughts came to me overnight, particularly regarding dated backups.

Currently, the yyyy-mm-dd date is added to the end of the folder/file names.
Consider when there are lots of dated settings backups and perhaps project backups in the same backup folder.
In File Explorer, when sorted by Name, backups for the same date do not cluster together but are widely scattered.
Even sorting by Date Modified does not help as the ProgramData & AppData folders keep their original date.
Sorting by Date Created would work but that is not shown by default in a File Explorer Details View.
That makes it difficult to select a dated group of folders/files when they need to be copied elsewhere.
Would it be better to add the yyyy-mm-dd date near the start of the name, e.g. fh_2022-04-08_

Alternatively, save all the backup files in a single dated folder, e.g. fhBackups_2022-04-08
Thye would then cluster together in File Explorer in date order whichever View is used.
The saved folders/files would be named exactly the same as for undated backups, i.e. no date in the name.
It would be easier for users to copy a particularly dated set to somewhere else.
I suspect the batch scripts would be identical in all cases, except for an initial CD command to choose the dated folder, which might even be governed by a prompt.

Other Suggestions:
In the C:\ProgramData\Calico Pie\Family Historian\Plugin Data\ folder the .ini filename should be derived from the Plugin Name but omitting the version suffix. That ensures it does not conflict with any other plugin that might be named fhWindowsBackupScripts and choose to use the same .ini file strategy.

Another caveat is that the batch script Reg export & Reg import commands are not supported in Wine/Crossover, etc.
I'm not sure about the robocopy command or zip file commands.
So the whole process may not be available to Wine/Crossover emulator users.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ADC65
Superstar
Posts: 457
Joined: 09 Jul 2007 10:27
Family Historian: V7

Re: Backup and Restore FH Settings via Windows - new plugin

Post by ADC65 »

I get the feeling that the underlying theme of Mike's suggestions here is that as soon as you start down a plugin path, especially one that isn't simply running inside FH on FH data, you start to encounter all sorts of different scenarios to deal with, and differing levels of user abilities. This makes the plugin ever-more complex with various niceties and options added, leading to an unsustainable maintenance overhead as user requests pile up.

I believe the original thrust of this idea was to make a simple script for at least medium-savvy computer users. Less knowledgeable users could continue to use Mike's comprehensive Backup and Restore plugin. It's not for me to decide what authors want to do with their work, of course, but I think I will use the simple commands Mark has provided, add them to a batch file, and run that directly when I want to do a Backup. I can then just copy the folder with a prefixed date to a different location. If and when I want to do a restore, I know how to do it, basically by reversing the above.

I guess what I'm saying is that I prefer the bare-bones script originally proposed and I think I will go back to using that, with a bit of manual renaming and moving afterwards. Thank you both for your efforts in continuing to help the user community, and let's hope that CP eventually build the capability into FH!
Adrian Cook
Researching Cook, Summers, Phipps and Bradford, mainly in Wales and the South West of England
User avatar
mjashby
Megastar
Posts: 719
Joined: 23 Oct 2004 10:45
Family Historian: V7
Location: Yorkshire

Re: Backup and Restore FH Settings via Windows - new plugin

Post by mjashby »

FWIW the best option for users of Wine/Crossover etc. is to regularly backup (i.e. Export) the application's Wine Prefix, which includes all settings stored in Appdata, Registry files etc. That option provides a 'protected snapshot' which can be reimported to restore an 'as was' version of the original installation with all settings in place, or which enables the application be moved to a different computer if necessary. How often this 'backup' process is needed is largely dependent on how often application/plugin updates are applied or if any user settings/preferences etc. are changed significantly.

Backup regimes on Mac's and Linux PCs can also be easily tailored to ensure Wine/Crossover Prefix Folders are included.

Mervyn
User avatar
ColeValleyGirl
Megastar
Posts: 5464
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by ColeValleyGirl »

satyricon wrote: 08 Apr 2022 10:53 This makes the plugin ever-more complex with various niceties and options added, leading to an unsustainable maintenance overhead as user requests pile up.
I have often thought that trying to be all things to all users makes a plugin hard to use and hard to maintain. I too will probably stick with the barebones approach...
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

Good ideas all, so thanks for those.

One very significant advantage for me as author is that this plugin is only about 5% of the amount of code in the original version, as Windows itself is doing all the complicated stuff. Of that 5%, by far the biggest component is making the UI look and feel as it does. The block of code that defines what the scripts look like is relatively small and easily modified.

I think I can address all of these excellent point with only modest changes.
  • Its “out of the box” configuration does what Adrian and Helen want - generates simple scripts that the user can run with. That type of user may only ever run the plugin once.
  • I prefer dated subfolders over file prefixes, simply because I think they look better, so that is a good change.
  • To cater for the inexperienced user, I was considering an “Update” button in the restore frame that prompts the user to select the folder containing the backup. It then updates the path specified in the restore script to match that location. For example, a USB stick is F:\ on my desktop and D:\ on my laptop, so the script is modified without the need for a manual edit. Restoring then becomes a simple matter of running the plugin, selecting this option, closing FH, and running the script. IMO, that’s no more difficult to describe for the PC novice than the existing process, which has the potentially confusing extra step of the Registry restore.
  • Given how simple the plugin is, I think it will be ready for the store fairly quickly without having to wait a long time accumulating user testing. At that point, I’ll change the name of the saved ini file to match the plugin store name.
It might take a few days to implement all that, plus native Windows zipping, but I think it makes sense to do it all before issuing an update.
Mark Draper
User avatar
Jane
Site Admin
Posts: 8507
Joined: 01 Nov 2002 15:00
Family Historian: V7
Location: Somerset, England
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Jane »

There is another advantage with this method, which I do not think has been mentioned.

I suspect the backup can be restored on to a "barebones" PC, without Family Historian installed and if a later version of FH is installed the automated "first run" routines in FH will do any element upgrades as needed.
Jane
My Family History : My Photography "Knowledge is knowing that a tomato is a fruit. Wisdom is not putting it in a fruit salad."
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore FH Settings via Windows - new plugin

Post by tatewise »

That is an interesting idea Jane.
The backups don't include the Registry HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Calico Pie\Family Historian\2.0 keys that hold the program version and build numbers.
So I wonder what the FH installation process will make of that and whether the other Registry keys will get overwritten or not.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

One reason that I’m pursuing this now is how easy my recent backup/restore was after a system upgrade (albeit the conventional way around, installing FH first and allowing it to activate before running the restore script). The key Mike described seems to hold the activation record as well, as messing with that upsets activation.
Mark Draper
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore FH Settings via Windows - new plugin

Post by Mark1834 »

Ready to go now with a series of relatively minor mods:
  • The zip option now uses Windows PowerShell options rather than 7-Zip. It takes a lot longer to process the files, but doesn’t need any extra software to be installed.
  • The template scripts use the Windows environment variable %USERPROFILE% rather than an absolute path, making them much more portable when restoring back to a different PC with different user path name. For example, my desktop is C:\Users\MarkD, and my laptop is C:\Users\Mark Draper. I didn’t set it up that way, but they were the Windows defaults (both new builds within the past few months). It appears user name truncation on some systems is a long-standing Windows bug that persists even into W11.
  • Dated backups each have their own subfolder of the form fhBackup_2022-04-14.
  • If you run a backup directly within the plugin, the command window stays minimised unless the user selects the option to view output. No change in practical function, but I think it looks neater.
  • The Help button is mapped to the actual Plugin Store help file. It’s only a brief placeholder description at the moment, but I’ll flesh it out before submitting the plugin.
  • The ini file has been renamed to match the plugin name. That means it will not inherit settings from the previous version, but all future versions will be fully compatible.
  • I decided against having an additional “update” button, as that is an unnecessary complication. Instead, the plugin silently checks that the selected path on loading actually exists on the host PC, and that any fhRestore.bat file in that folder is calling a valid location.
It’s worth checking my objectives to see if the plugin is doing what was planned -
  1. To provide more experienced PC users with template backup and restore scripts that they can adapt for their own purposes to suit whatever backup strategy they choose. As I commented before, that type of user may only use the plugin once.
  2. To provide all FH6/7 users with a simple to use and reliable method for backing up FH settings from within the plugin, irrespective of language and location. Realistically, users who are happily using the current plugin and are not interested in automatic backup will probably continue with it while it is still supported, but this is an alternative for new users.
  3. To provide a restore process that is no more complex than that of the current plugin, and readily explained to users. It’s worth bearing in mind that most users run far more backups than restores, and restores are most frequently used to transfer or copy settings to a replacement or additional PC, so by definition not novice PC users.
I think I’m still on track, but over to you to try and break it...😉

[Attachment deleted as the plugin is now in the store]
Mark Draper
Post Reply