* Backup and Restore Family Historian error messages

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
avatar
ELM
Platinum
Posts: 35
Joined: 24 Sep 2003 14:44
Family Historian: V7

Backup and Restore Family Historian error messages

Post by ELM »

I am attempting to use the backup and restore family historian plugin to transfer my work between two computers. When using the plugin to create a backup, I receive an error message indicating that some of the icons are missing. This is accompanied with a text document that states the missing files (note attached screenshot). The error message suggests moving the icon files into the folder generated by the backup, however closer inspection of the new icons folder shows that all the missing icons are in fact in place.

What do I need to do to complete the backup and subsequently transfer this to the other computer?
Attachments
Screen Shot 2021-03-26 at 13.55.03.png
Screen Shot 2021-03-26 at 13.55.03.png (129.89 KiB) Viewed 5857 times
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Please compare the path to the Icons in the ImageList.txt file and the current FH Icons folder path.

The current FH Icons folder path in a standard Windows configuration is:
C:\ProgramData\Calico Pie\Family Historian\Icons\...

The ImageList.txt file paths are completely different and unlikely to migrate to a new PC:
C:\Documents and Settings\All Users\Documents\Family History\Family Historian\FH Icons\...

C:\Documents and Settings\All Users\ is derived from Windows XP so I guess you set up these Icons long ago.

If you view your Custom Ancestors & Descendants diagram do those Icons get displayed?
Whether they do or not you need to go through its Diagram > Options > Boxes tab Conditions and for each Box Feature change the Icon setting to use the current FH V7 Icons because your external folders are no longer allowed.
You will almost certainly need to do that for every type of Diagram for which you have set Icons, not only so they will migrate to the new PC, but because FH V7 uses a new Icon file type and the .bmp files are no longer well supported.

The last entry in the ImageList.txt file relates to an Icon defined in the Standard Diagrams.
Its file path is in a Dropbox sub-folder and that needs similarly resetting as above.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
KMTV
Gold
Posts: 18
Joined: 28 Dec 2020 18:32
Family Historian: V7

Re: Backup and Restore Family Historian error messages

Post by KMTV »

Hello Mike,

I'm trying to help ELM with this issue.

Questions :

What does the "CUSTOM\ANCESTORS & DESCENDENTS.FHDX" Refer to, and where can it be found?

Having found the folder that you specified [C:\ProgramData\Calico Pie\Family Historian\Icons\...] we found that the icons in use by the diagram were originating from there. Is it possible to find out or set the exact file path used by an individual icon, as I suspect that the confusion has been caused by the icons becoming duplicated?

Thanks
N.B. I'm not actually a user of any kind of genealogical software!
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

CUSTOM\ANCESTORS & DESCENDENTS.FHDX refers to the file in:
C:\ProgramData\Calico Pie\Family Historian\Diagrams\Custom\Ancestors & Descendants.fhdx

In FH V7 open that Diagram via Charts> Custom Diagrams > Ancestors & Descendants.
Does it display any Icons at all? I suspect it doesn't.
Anyway, FH V7 only supports Icons in C:\ProgramData\Calico Pie\Family Historian\Icons\ and nowhere else.
(The reason for that is to allow FH Diagrams to migrate to a new PC where the Icons will be in the same folder.)
Also, those old type of .bmp Icons are not well supported, and they all need to be replaced.
So use Diagram > Options > Boxes and for each Condition click Edit Features... and choose an Icon.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
KMTV
Gold
Posts: 18
Joined: 28 Dec 2020 18:32
Family Historian: V7

Re: Backup and Restore Family Historian error messages

Post by KMTV »

Hi Mike,

Progress! Between us we reassigned all the missing icons from the custom Ancestors and Decendents diagram.

After removing the Custom\Ancestors and Diagrams.fhdx and ._Ancestors and Diagrams.fhdx files, a backup was successfully made with no error messages.

This is still using .bmp files for now. What format should the icons be in?
N.B. I'm not actually a user of any kind of genealogical software!
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Sorry, you have completely confused me.

Have you added some of your own .bmp Icons to the FH V7 Icons folder and assigned those to the diagram?
Virtually all the FH V7 installed Icons are .emf files so it would be better to assign them.
The Number Labels are ideal for Census years and Miscellaneous has document-B / M / D files for BMD Certificates.

If you have removed the Custom\Ancestors and Diagrams.fhdx file from the C:\ProgramData\Calico Pie\Family Historian\Diagrams folder then that has deleted your diagram customisations. So I'm not clear what you have removed.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
KMTV
Gold
Posts: 18
Joined: 28 Dec 2020 18:32
Family Historian: V7

Re: Backup and Restore Family Historian error messages

Post by KMTV »

I'm not sure what the confusion is?

The Icons folder is predominantly populated with .BMP files. There are some .EMF files which do not appear to work properly - note attached image.
Attachments
IMG_1724.jpg
IMG_1724.jpg (315.34 KiB) Viewed 5730 times
N.B. I'm not actually a user of any kind of genealogical software!
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

OK, I see.
About 50 of the .bmp files that are held directly in the Icons folder are inherited from FH V6.
FH V7 only installs files in the 9 folders DNA, Flags, Gender_icons, Letters, Miscellaneous, Monochrome, Number Labels, Popular & Shapes, and apart from Gender_icons, they are all .emf files that may need a suitable image viewer such as IrfanView.
ELM must have populated the Icons folder with all the other .bmp files & folders before installing FH V7 because the Birth/Marriage/DeathCert.bmp and Census**.bmp icons are not among the standard installed FH icons.

What confused me was the icon file paths in ImageList.txt were nothing like the standard FH installation Icons folder, so I assumed that no such Icons existed there and thus unsure how you had assigned them to a Diagram.

If you are happy with the way your .bmp icons are displayed in Diagrams then go ahead with the migration to the new PC.
Some users have found the size & position of .bmp Icons unsatisfactory so have switched to .emf Icons.
If ELM runs into problems on the new PC then that can be resolved later.

I also was unsure about how you have 'removed' the Custom\Ancestors and Diagrams.fhdx file without destroying the custom Diagram in FH.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
ELM
Platinum
Posts: 35
Joined: 24 Sep 2003 14:44
Family Historian: V7

Re: Backup and Restore Family Historian error messages

Post by ELM »

Thanks Mike
As you say I installed the bmp icons on V6. The Custom\Ancestors and Diagrams.fhdx file was from then and had been abandoned after saving the diagram with a different name. The problem with the plug-in is now solved. The other problem with saving and restoring custom diagrams is now another thread.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

OK, thank you for clarifying, it is always a bit disturbing when someone says they have deleted something but everything is still working as required. You know you have substituted an alternative but I don't. I wish you well on your new PC.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
johnmorrisoniom
Megastar
Posts: 901
Joined: 18 Dec 2008 07:40
Family Historian: V7
Location: Isle of Man

Re: Backup and Restore Family Historian error messages

Post by johnmorrisoniom »

I have purchased another license for FH7 so that I can use a 3rd computer, so went to use Backup and restore on my main computer, in order to restore to the new installation on my laptop but got the following error (BU&R v 3.0)

[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4452: not enough memory for buffer allocation
stack traceback:
[C]: in method 'read'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4452: in upvalue 'doCopyFile'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4493: in upvalue 'intCopyFiles'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5372: in function <[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5335>
(...tail calls...)
[C]: in function 'iuplua.MainLoop'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:2439: in field 'ShowDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5768: in function 'GUI_MainDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5808: in main chunk

The backup Folder is created containing 2 folders - Application (Empty) and Program Data (Contains some folders, but not all that should be there).

I use symbolic links for the entire Program data folder, so they are already on the laptop (Linked folder is on ONE DRIVE)
I think the plugin stopped while backing up icons
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Sorry about that John. Memory issues seem to be more of a problem with FH V7 Lua 5.3 than with FH V6 Lua 5.1.
There cannot be significantly more 'memory leaks' in this Plugin script as not much has changed recently.

As an experiment, I have set the memory management parameters that work well in Export Gedcom File.

So try the attached Backup and Restore Family Historian Settings plugin Version 3.0.1 Date 10 Apr 2021.
Last edited by tatewise on 09 Sep 2021 18:16, edited 1 time in total.
Reason: Attachment deleted as later version is in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
johnmorrisoniom
Megastar
Posts: 901
Joined: 18 Dec 2008 07:40
Family Historian: V7
Location: Isle of Man

Re: Backup and Restore Family Historian error messages

Post by johnmorrisoniom »

Hi Mike
Error after 131 files backed up.

[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4458: not enough memory for buffer allocation
stack traceback:
[C]: in method 'read'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4458: in upvalue 'doCopyFile'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:4499: in upvalue 'intCopyFiles'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5378: in function <[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5341>
(...tail calls...)
[C]: in function 'iuplua.MainLoop'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:2439: in field 'ShowDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5774: in function 'GUI_MainDialogue'
[string "C:\ProgramData\Calico Pie\Family Historian\Pl..."]:5819: in main chunk

Also tried different backup folder (Not in FH Projects folder) with same result. I did successfully backup when I first started using FH7, but have made a few changes since then.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

I don't think it is anything external. It is the internal memory (RAM) being used by the Lua interpreter.

I have made the memory garbage collection more aggressive and added a debug message when the error is detected.
It reports the error message, the copy operation, size of the file, memory in use, and the source and target file paths.
e.g.
not enough memory for buffer allocation

Backup File Size=318 Bytes Memory=770 KB

C:\ProgramData\Calico Pie\Family Historian\Flags\Flags.fha

E:\Backup\Family Historian\Program Data\Flags\Flags.fha

Click OK and it will proceed with the copying until the next error.
Note the files that fail to copy and manually copy them from source to target if you wish to complete the backup.

Attachment Backup and Restore Family Historian Settings plugin Version 3.0.2 Date 11 Apr 2021
Last edited by tatewise on 09 Sep 2021 18:17, edited 1 time in total.
Reason: Attachment deleted as later version is in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
johnmorrisoniom
Megastar
Posts: 901
Joined: 18 Dec 2008 07:40
Family Historian: V7
Location: Isle of Man

Re: Backup and Restore Family Historian error messages

Post by johnmorrisoniom »

Thank you Mike.
I have some seemingly very large home made emf icons which are causing the problem.
Screenshot 2021-04-11 130316.jpg
Screenshot 2021-04-11 130316.jpg (25 KiB) Viewed 5378 times
Screenshot 2021-04-11 130238.jpg
Screenshot 2021-04-11 130238.jpg (27.12 KiB) Viewed 5378 times
Screenshot 2021-04-11 130154.jpg
Screenshot 2021-04-11 130154.jpg (28.92 KiB) Viewed 5378 times
(There were over 30 failed files altogether)

But theses get copied across through symbolic links anyway.

The backup then completed OK.

Now to test the restore on the laptop
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Otherwise, the memory usage looks much the same as when I run the Plugin.
All the .emf FH Icons that I've looked at are only a few KB so why are yours so large, several hundreds of MB?
Is it possible to reduce their complexity and size?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
johnmorrisoniom
Megastar
Posts: 901
Joined: 18 Dec 2008 07:40
Family Historian: V7
Location: Isle of Man

Re: Backup and Restore Family Historian error messages

Post by johnmorrisoniom »

That is my next task. I am going to redo a lot of the homemade ones a different way to see if I can reduce the file size (They are a 2 digit number overlaid on the national flag, starting with the supplied emf of the flag. (I have to covert the number to a vector).
It just takes time to do them
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Interesting, given that the FH supplied Flag of Canada is only 2K and the Number Labels at most 7K.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore Family Historian error messages

Post by Mark1834 »

Just out of curiosity - why does the plugin use the Lua io library to copy files, rather than just os.execute and let Windows copy the files and manage memory. On the surface it seems a lot simpler and more reliable, or have I missed something?
Mark Draper
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Backup and Restore Family Historian error messages

Post by ColeValleyGirl »

Mark, os.execute is very system dependent; the lua io library less so.

There'll be another option when FH 7.0.4 comes out: a library which handles UTF8 chars in both file names and directory paths (using luacom)
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Mark, maybe that is a good idea, or use the fhShellExecute() function.
The advantage of fileTarget:write(fileSource:read("*all")) is that it is OS independent.

Do you know the os.execute(...) parameters needed to copy files?
They are OS dependant but since FH only runs in Windows that should be OK even for Crossover, PoM & Wine.
I don't suppose it is as simple as os.execute("Copy C:\\file1 C:\\file2")

Helen, how will FH 7.0.4 help with copying files? Is there a method using luacom?
I recognise that handling UTF8 in file paths is something I've been requesting for years and allows unrestricted file names but does not directly offer file copying.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
ColeValleyGirl
Megastar
Posts: 5465
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Backup and Restore Family Historian error messages

Post by ColeValleyGirl »

Mike, the library will support most common file operations, including copying. Yes, all using luacom.

http://pluginstore.family-historian.co. ... Utils.html
User avatar
Mark1834
Megastar
Posts: 2458
Joined: 27 Oct 2017 19:33
Family Historian: V7
Location: South Cheshire, UK

Re: Backup and Restore Family Historian error messages

Post by Mark1834 »

Yep, it's as simple as that :). You could even use os.execute() with a robocopy rather than copy command and generate a complete mirror of the ProgramData folder in the backup location with one line of code.
Mark Draper
User avatar
johnmorrisoniom
Megastar
Posts: 901
Joined: 18 Dec 2008 07:40
Family Historian: V7
Location: Isle of Man

Re: Backup and Restore Family Historian error messages

Post by johnmorrisoniom »

The restore to the laptop went smoothly and no problems.

My new Canada Census icons are only 3kb. I will make them all available to everyone when they are all done.
User avatar
tatewise
Megastar
Posts: 28341
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Backup and Restore Family Historian error messages

Post by tatewise »

Please remember this public Plugin Store plugin, like many of mine, must run in FH V5 and FH V6 and FH V7 and the user must not be required to install anything.

Helen, that rules out an FH V7 specific library unless I use its luacom features as an alternative in the plugin script for FH V7 and keep the existing Lua method for earlier FH versions.

Mark, so far I am struggling to get os.execute('Copy "C:\\file1" "C:\\file2"') to work but still trying things.

John, thank you for the feedback.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
Post Reply