Page 1 of 1

Find Duplicate Facts

Posted: 04 Dec 2021 16:21
by Kingy
Mike

I have merged 2 trees (don't ask, it was a nightmare) but I have ended up with duplicate facts on some people.

Born in x duplicated
Census in xx duplicated
Occup. x duplicated
Died in x duplicated

But I see your plugin doesn't find these. Is there a reason you haven't and is it easy to amend for me to do if it can be done.

Thanks
Paul

Re: Find Duplicate Facts

Posted: 04 Dec 2021 16:23
by Kingy
Sorry just found reason myself, very subtle typing difference between the 2 entries.

Ignore will do it manually as I find them.

Thanks

Paul

Re: Find Duplicate Facts

Posted: 04 Dec 2021 18:06
by tatewise
The only two items the plugin needs to identify duplicates is the Fact Name (obviously) and the Date.
Everything else can be disregarded by removing all the tick options.
So I guess the "very subtle typing difference between the 2 entries" is in the Date somewhere?

If it is something that other users could do by mistake, is it possible the Plugin could disregard those subtle differences and identify the Facts as duplicates to the benefit of all? Can you reveal what the subtle differences were?

Re: Find Duplicate Facts

Posted: 04 Dec 2021 19:47
by Kingy
I have attached an example which isn't being picked up.

Paul

Re: Find Duplicate Facts

Posted: 04 Dec 2021 20:44
by Kingy
In the Census duplicate the Place is Town, County, England on one and just Town and County on other.

The occupation is exactly the same in both entries.

Paul

Re: Find Duplicate Facts

Posted: 04 Dec 2021 20:57
by tatewise
Firstly, have you followed the Plugin advice and sorted all Facts into Date order? Tools > Re-order Out-of-Sequence Data...
Just because they are shown in Date order in the Facts tab means nothing.
The Facts tab always shows Facts in Date order regardless of database order.
To see what I mean, look at them in the All tab which is shown in database order.

Did you clear the tick options in the Plugin so that the Fact Place (PLAC) and other fields are disregarded?

The two Census facts on 30 Mar 1851 should get detected.

I can see possibly why the four Facts on 2 Apr 1871 are not detected if the two Census facts and the two Occupation facts are not adjacent.

The plugin works by detecting adjacent Facts with the same Label and the same Date.
So if the order is Census, Occupation, Census, Occupation then they are not spotted. I can investigate that rarety.
Almost any other order will result in at least one pair being detected and when resolved the other pair will be detected.

Re: Find Duplicate Facts

Posted: 04 Dec 2021 21:27
by Kingy
Appreciate your help.

I wasn't aware of the fact the dates needed sorting via the sequence tool, which I ran and it found loads of out of sequence, so that helped, and I ran the find duplicate facts again and it found loads.

Strangely, the example I posted above wasn't one that changed. I understand what you are saying about not being next to each other, but that doesn't explain why the 1851 Census was unchanged.

I then moved the order manually to put the 1871 Census next to each other and the 1871 Occupation next to each other and ran again, and it found the duplicate occupation, but again not the Census.

Hope that is clear.

Paul

Re: Find Duplicate Facts

Posted: 04 Dec 2021 21:31
by Kingy
sorry forgot to add I did remove all ticks before running.

Paul

Re: Find Duplicate Facts

Posted: 04 Dec 2021 21:45
by tatewise
Thank you. I've double-checked the Plugin script and discovered it does check both the Date and Place of adjacent Facts with the same Label, despite the Fact Place (PLAC) option being unticked. :oops:
I will look at that tomorrow and hopefully improve its ability to detect duplicates.

Re: Find Duplicate Facts

Posted: 05 Dec 2021 13:16
by tatewise
I've updated the plugin, and as a bonus, it now does not need Facts to be sorted, and should find any duplicates.

If all the tick options are removed then duplicates only need the fact Label, attribute Value, and fact Date to match.

Try attachment Find Duplicate Facts plugin Version 1.1.1 Date 05 Dec 2021.

Re: Find Duplicate Facts

Posted: 05 Dec 2021 20:03
by Kingy
Many Thanks Mike

That did the trick perfectly.

I really appreciate your help.

Paul