* V6 Backup very large files

[rkanaar]

I have just upgraded to V6 from V4. When I closed V4, the backup or skip window appeared (as in V6), and when I chose backup, it created the zip file almost instantly and closed FH. In V6, the backup never completes, it tells me it is adding file xx of xxx, and then just hangs. I am backing up to exactly the same destination as I used for V4. Has anyone any ideas please. I have not changed any of the default backup settings, except to create the default drive.

[tatewise]

FH V5 & V6 have three Backup modes, whereas FH V4 only had one Backup mode for GEDCOM only.
The two new modes are Medium that includes the GEDCOM and Media images, and Full that includes the entire Project folder, GEDCOM, Media & Public, etc.

FH would appear to be having a problem writing to your new 'default drive'.
What is the Tools > Preferences > Backup > Default Backup Folder setting?
Are you sure that folder exists and has write access?

See glossary:backup_and_recovery|> Backup and Recovery for more explanation.

[rkanaar]

Hi Mike

the default is set to F:, which I selected using the 'select' button. The drive is fully open, and is used by my laptop regularly during the day, and is exactly the same as I used for V4. The other settings I have are as per attached screen shot. I notice in my backup file that when I upgraded it generated a ' Pre Upgrade full master.zip', so backup must have worked as part of the upgrade, but nothing since, and as I am updating projects regularly, this is worrying.

thanks

Rob

[tatewise]

What happens if instead of F: you use F:\ or add any existing folder name?

[rkanaar]

Hi Mike

tried amending default drive and/or folder name - no difference. Interestingly the 'adding files to backup' progress hangs at 134 records out of xxxx, whichever project it is I am backing up. When it reaches 134, the close/backup window disappears, and FH remains open. If I try to close FH using the standard windows program close 'X', FH application header states 'Family Historian............' (Not Responding), then the standard windows programme not responding window opens with windows trying to fix the problem. I normally then use windows task manager to close FH. This time, whilst typing this response, I have left FH open, and the progress dialogue has re-opened, and now climbed to 338/xxx, and hung for about 10 minutes now, with the windows 'running' icon just going round, and FH application header still stating 'Not Responding'. Something really weird happening here, now the taskbar FH icon will not open the FH window, have to minimise other apps on top of FH. Running on Windows 7.

Rob

[Jane]

Do you have any very large attachment on your Project Videos or Audios?

[rkanaar]

Further update - have checked the backup folder on F drive - new zip file header created, but will not open, file not recognised, so presumably FH has created the file name, but nothing in it?

[rkanaar]

Hi Jane

I do not have any video or sound files. I have the usual images, pdf files, etc, but I have not added any media since upgrading.

Rob

[tatewise]

Plase check the Tools > External File Links and the Media Widow to see if any Media files are marked with a large X.

Are there any exceedingly large files anywhere else in your Project folder, not just in the Media folder, but in the Public folder, etc?

[johnmorrisoniom]

I have found that the new file "ThumbCache.dat" in the Cache folder in my main project is almost 1.5Gb. This file only came in with vers 6 . Could this also causing problems with a full backup?

[rkanaar]

In the external file links there are 12 red crosses - 1 txt file, 1 htm file and 10 jpg files - one of these jpg's is a 4.3 mb map, others are register page copies, so typically 5kb.

[rkanaar]

Have now fully checked the project media folder in windows - it has some huge tif and zip files, which do not exist in my master media reference files, used to store original files, nor in the tools/work with external links, nor in the media records list. I recognise the file names, but have no idea how they are in the FH project folder, as I do not have any links to them that I can find. They also appear in projects that do not even have these people in them. If these are the problem, how have they been created in FH, and why did this not happen with V4 backup please?

Is the simple answer to now go through all my project folders in windows and delete these large files?

[tatewise]

John, that cache file is certainly included in the Medium & Full Backup. If you perform a Medium Backup do you get similar symptoms? If so, and you temporarily move the file out of the Project folder, do the symptoms cease?

Bob, check the cache file mentioned above, and try the corrective action of moving it out of the Project temporarily.

The red X usually means the file path link to the file is incorrect/broken. Select one of them and click the Go To button to open the Media record in the Media Window where it probably displays a rectangle with a X. Double-click in the rectangle and the faulty path is shown in red, which when clicked gives options to find the file and correct the path. Repeat this for all 12 broken links.

Only files in the Media sub-folder are likely to be linked to FH Media Records. The files in the other folders within the {project-name}.fh_data folder such as Books, Cache, Charts, Plugin Data, Settings were probably created by FH itself. Files in the Public folder are often Exports or Save As... from FH initiated by you. However, there is nothing to stop any files being copied into those folders by you or a program you use.

There would never have been a problem with FH V4 Backups because it only included the GEDCOM file alone, and no Media or any other folders.

If the very large files are not required then they can be deleted or at least moved out of the Project folders.
It would be useful to temporarily move them in and out of the Project folders to see if the Backups then fail/succeed. If they prove to be the cause then details of their size and type will help Calico Pie fix the problems.

[jimlad68]

Rob, don't know if this is anything to do with it, but you mentioned "default is set to F:". Some time ago MS altered Windows 7 so that I could no longer save files in the root of a drive (say if you want to save a file in the root you get the message "you do not have permission to save in this location"), however, I note you also say "tried amending default drive and/or folder name - no difference", so if you tried a non root folder this aspect should not be a problem.

[rkanaar]

Hi Mike

moved out the thumbcache - backup started faster, then suddenly stopped, and FH went into Not Responding mode. Had to close through task manager. Checked the windows explorer, FH had re-created the thumbcache - deleted the new one, and replaced with the original.

Have been checking and correcting/deleting media file X's - have done this for all records except one. It is a file name I do not recognise, the work with external file links shows a persons name I know, but it is spelt wrong, and searching for this name spelling in FH shows no records spelt this way. Doule clicking the persons name in the Media List, and opening the Multimedia object window for this record, under the 'All' tab I can see the record ID. Searching that record ID, it is for a person with a totally different name, and they have no media files attached. Going back to the media list window and checking the 'Links' tab, I have no links. Finally found the details in the main media records link by jumping to it from the external media view, used the main edit menu and delete, and this has removed the obscure media.

When I saved and tried to backup, it instantly failed 'Not Responding'.

Then removed thumbcache again - got an error message - unable to open thumbcache, skip this file and continue? - Clicked yes, backup still hangs, and FH goes to 'Noe Responding', and fails to complete. Strikes me the media file backup is the issue, I will have a go at changing some of the backup settings after Christmas, getting it in the neck from the other half for spending all day on FH

[tatewise]

What about the "huge tif and zip files" elsewhere in your Project that you mentioned above, and I suggested moving out temporarily?

Glad to hear you fixed all the rogue Media links OK. It is much easier in FH V6.

I suspect the ThumbCache idea may be red herring.

[rkanaar]

Mike

deleted all the huge files, they were tif files, and the media links were to the jpg versions, so have no idea why tif versions were even in the folder, also a zip file, which was only ever used in sending images to a friend through email, so never even in a project!!!

Saved and backed up - instant error message of 'not Responding', ignored this, and left FH, backup then ran, and the taskbar FH icon allowed me to access FH window, unlike before. Backup has completed, and the backup file opens properly.

When I next save and backup, will the backup be incremental, or will FH create a new file? An incremental backup would be brilliant, as otherwise, this is going to be a pain, and also seems to show that regardless of the backup medium's capacity, FH seems to have issues with backing up large files.

I am also intrigued as to why backup says on progress xxx of ~500, when I have about 1000 families in the project, and many more people. What does this count represent please?

thanks

Rob

[tatewise]

Looks like huge files are a problem that Calico Pie need to fix.

Backups are NOT incremental, but you only need to do Full or Medium backups when new files are created. Often a GEDCOM only Backup is enough.

The progress is reporting the number of files being backed-up.

[johnmorrisoniom]

I think it is the Windows archiving system itself that has always had problems with large files and archives. I regularly get zip files from one of my suppliers who recommend using winrar or winzip to extract the contents because windows can't cope.

[rkanaar]

Mike

many thanks for all your help, all projects now cleared of large files and fully backed up. Not only do Calicoe Pie need to look at large file backup facilities, but if as you rightly say a full backup is only needed when changing media files etc, the save/backup window on closing must be changed, as otherwise the only way to change backup levels is in the menu, and I do not really want to be going here every time, and often I do not know what will be changed each time I update a project, so a reminder at 'close' time with backup options is what is really required. I will work out suitable wording for the wish list.

Rob

[tatewise]

It is already there in Tools > Preferences > Backup.
Just select Backup Button Action: Let Me Choose Backup Type.

[deepdiver]

Hi all

I know this post was started about 3 years ago (I looked for the problem I had before posting my query) but I think it is relevant to what was said before. I am posting it to add to the comment on the last post about the ability of FH to handle large backups, in Windows (in my case Win 10, ver. 1709).

I am starting to get this problem with my backups, although they do seem to complete, eventually. My projects are my one-name study, and so there do tend to be a lot of files, especially media files. These are usually images of documents for sources noted, rather than photographs of people, and may sometimes therefore be of a high(ish) resolution.

I find that if I do an Entire backup, it starts well enough, says it is backing up xxx/369 files, gets to about 367 then hangs. Windows then pops up a message saying Not Responding. If I leave it for a minute or two, the box disappears, it appears to complete, and Family Historian pops up its own box message up saying Backup Completed.

I had previously checked and removed any unlinked media files.

Just for information, anyway.
Andrea

[tatewise]

Andrea, have you investigated all the files in your Project subfolders?
You need to look, not just in the Media subfolder, but all the subfolders, and if performing a Full Backup then that includes the Public subfolder.
Are any files particularly large, especially those in the Cache subfolder?

[deepdiver]

Mike

Aha, thank you. ThumbCache was not so big but...

Public folder WAS 551MB containing several other old Gedcom files, but I've now deleted these.

And it backs up OK now, thank you. I had not realised that previous exports of gedcoms would end up here.

Thanks for your helpful and prompt response, it is appreciated.

Andrea

[tatewise]

Yes, typically exported GEDCOM files default to Public folder.
If there is nothing of great importance in Public folder then use Medium Backup instead.
Also while talking about backups, don't overlook Backup and Restore FH Settings Plugin.