* Exception Reports

[ColinMc]

Last night at 8 pm when I exited FH, a message I'd never seen before gave me the following text file

Lines marked "EXCLUDED:" indicate data which has not
been loaded, and explain why it wasn't loaded.

Indented lines marked "EXCLUDED BRANCH LINE:" indicate
data lines which are not necessarily invalid in
themselves but which have had to be excluded as a
consequence of excluding other lines.

Lines marked "INFO ONLY:" indicate data that has
been successfully loaded, but where special action
was needed to load it (e.g. GEDCOM errors detected
and automatically corrected by Family Historian).

---------------------------------------------------------
EXCLUDED: link to non-existent Source record (22)

EXCLUDED: link to non-existent Source record (1150)

EXCLUDED: link to non-existent Source record (1150)

EXCLUDED: link to non-existent Source record (327)

EXCLUDED: link to non-existent Source record (327)

EXCLUDED: link to non-existent Individual record (386)

....... continues

It suggested, that I should validate the file which I did, and its said the file was OK.

This morning I got a similar (but very different) error on opening the file.

Family Historian Exception Report - Wednesday 21 Apr 2021
=========================================================

Lines marked "EXCLUDED:" indicate data which has not
been loaded, and explain why it wasn't loaded.

Indented lines marked "EXCLUDED BRANCH LINE:" indicate
data lines which are not necessarily invalid in
themselves but which have had to be excluded as a
consequence of excluding other lines.

Lines marked "INFO ONLY:" indicate data that has
been successfully loaded, but where special action
was needed to load it (e.g. GEDCOM errors detected
and automatically corrected by Family Historian).

---------------------------------------------------------

Record Type=Individual. Gedcom Id=I2. Record Number=2.
l.354 - EXCLUDED: invalid line : "2 _SHAR "
l.355 - EXCLUDED BRANCH LINE: : "3 ROLE Executor"
l.356 - EXCLUDED: invalid line : "2 _SHAR "
l.357 - EXCLUDED BRANCH LINE: : "3 ROLE Executor"
l.382 - EXCLUDED: invalid line : "2 _SHAR "
l.383 - EXCLUDED BRANCH LINE: : "3 ROLE Informant"
l.384 - EXCLUDED BRANCH LINE: : "3 SOUR "
l.385 - EXCLUDED BRANCH LINE: : "4 QUAY 3"

Record Type=Individual. Gedcom Id=I3. Record Number=3.
l.413 - EXCLUDED: invalid line : "2 _SHAR "
l.414 - EXCLUDED BRANCH LINE: : "3 ROLE Informant"
l.459 - EXCLUDED: invalid line : "2 _SHAR "
l.460 - EXCLUDED BRANCH LINE: : "3 ROLE Executor"
l.461 - EXCLUDED: invalid line : "2 _SHAR "
l.462 - EXCLUDED BRANCH LINE: : "3 ROLE Executor"
l.465 - EXCLUDED: invalid line : "2 _SHAR "
l.466 - EXCLUDED BRANCH LINE: : "3 ROLE Beneficiary"
l.467 - EXCLUDED: invalid line : "2 _SHAR "
l.468 - EXCLUDED BRANCH LINE: : "3 ROLE Beneficiary"

...... continues

Again validation was suggested, this time, the following message was given.

Family Historian Validation Report - Wednesday 21 Apr 2021
===============================================================

27 media items(s) not stored within a Media record. Recommendation is that all should be. 27 of these have been corrected.

In some ways, I'm not over-concerned as I have a backup from 3:25pm yesterday afternoon, and 4 from the day before, so it should be possible to revert to a good file without too many issue.

I am a bit concerned though as to what caused the problem last night.

To be completely fair to CP, I have NEVER had any kind of data error in the 3 years or so that I have been using the software. Any suggestions as to what might have caused the problem so that I can try to avoid any repetition.

Incidentally, I have run MS scans, and SMART disk checks to make sure my hard disks are OK, and they passed all the tests.

[tatewise]

That is certainly very unusual, and working backwards...

I think the last message regarding 27 Media is triggered by Local Media Objects that FH v7 wants to be in Media Records.
However, it is inexplicable why that was not reported by the previous Validation commands.

The excluded shared Fact Witnesses would not raise a Validation error because they have been deleted from the Project.

Similarly, the excluded links to non-existent Souce records would not raise a Validation error as they no longer exist.

I think the Validation command operates on the FH Project data in memory rather than the GEDCOM file.
Perhaps, the Exception Reports are driven by reading the GEDCOM file, in the first case after saving it, and the second case when loading it. That might indicate a serious problem with the Windows filing system not reading/writing files reliably.
Is there anything else that reads/writes those GEDCOM files, such as cloud storage (e.g. OneDrive), backup tools, anti-virus tools, etc, that could upset the read/write process?

It sounds like it should be reported to CP if only to understand what mechanisms trigger the Exception Reports.
Are they derived from reading a GEDCOM file as I suspect, or something else?

If you restore your Project from Backup it will create a new Project with a (2) suffix so the original Project is retained.
If you repeat the processes that led to the Exception Reports do they arise again?

[ColinMc]

Thanks for the help. I'm out all day so will try to answer the points tonite.

[ColinMc]

Well its certainly not a happy project when I opened it this evening.

I can't remember how many individual records I had (I'm guessing it was approx 2,500), but tonight I have 20. I have zero family records, zero sources and 27 media records - presumably from the quantity and time stamp - the 27 referred to in the validation message this morning.

FH crash 2.jpg (305.09 KiB) Viewed 2724 times

Is there anything else that reads/writes those GEDCOM files, such as cloud storage (e.g. OneDrive), backup tools, anti-virus tools, etc, that could upset the read/write process?

I keep most data locally, so I run my cloud storage app - MS OneDrive - on demand, and it has not been run for approx 2 weeks. As I back up every time I exit FH to an external drive, I run back-up to another external drive on demand as well. My back up routine stores deleted files as well as the current versions, and on checking, it looks like I have 71 data backups (there may be a couple of dupes there). Should be enough!!!

AV tool (Avast) does run permanently - but it runs all the time, and has never given me any issue.

I've just run CrystalDiskInfo again and all drives report Good. Speccy gave me the clear test result this morning.

Anyway, I'm now going to try restoring from backup, and reporting to CP.

[tatewise]

Whatever process upset things, it certainly made a mess
Maybe CP can throw some light on the problem.

[ColinMc]

My 3pm restore seems to have restored perfectly. I've opened and shut the file several times.

I will try and re-create the steps I took yesterday, but it will be almost impossible to do so.

I had a really busy day yesterday and probably used AS to create many new records
mainly from census, and marriage records. Also entered quite a few Electoral Roll records using FH. I suspect I added approx 100 records of various types. I can probably identify most but I cannot be certain.

I believe the last thing I did was to add a Canadian Marriage record in AS with a 100% custom built Auto-Template, built by me from scratch. I have just recreated that record, and FH seems happy with it. Opening and closing without issue. I did have a worry that that might have been a contributory factor but it looks OK.

[NickWalker]

Remember that, unless you've changed settings, AS backs up your file every time an entry is made (into documents\ancestral sources\backups) so you can always use those backups to recover too.

[ColinMc]

Thanks Nick,

I had forgotten that. I have 9 from yesterday - 4 of them after my last FH back-up, and 1 appears to be from just before/after I had problems.

I never can get my head around MS file dates/times. This one says it is created at 20:08 yesterday, but modified yesterday at 15:53! ie 5 hours before it was created.

2021-04-21_21-40-27.jpg (139.74 KiB) Viewed 2682 times

That file seems to open fine (I used a copy to maintain the original in case CP want it to investigate), and the last source I have showing there is timed at 15:52. The bulk of my records seem to be present & correct. Need to check some of the last edits I did.

2021-04-21_21-47-15.jpg (396.84 KiB) Viewed 2682 times

[NickWalker]

In case it isn't clear, the file name indicates the time the copy was made (which is part of the save process by AS) so, for example:

Backup 2021-04-20_20-08-32 Colin.ged

means the file was created in 2021 on April 20 at the time: 20:08:32

[tatewise]

I suspect the Modified date-time is when the original GEDCOM file was last updated.
When AS copies it as a backup that does not count as an update so the Modified value is not changed.
The new backup copy has a Created date-time for when AS created the copy.

[NickWalker]

Yes that's correct Mike.

[ColinMc]

Response from CP is

It sounds as if you might have had a disk error or two and the file has become damaged.

They advise disk checking.

I had already run three tests, I've now run a fourth different test and each one reports the disk to be in good (perfect according to one software) condition.

I've had no issues with any other software (or data), so I'm not convinced it is a disk issue, but obviously I cannot rule it out (or in). I'll keep an eye on things over the next few days and weeks, and ensure I have even more backups!