* Find Duplicate Facts

For users to report plugin bugs and request plugin enhancements; and for authors to test new/new versions of plugins, and to discuss plugin development (in the Programming Technicalities sub-forum). If you want advice on choosing or using a plugin, please ask in General Usage or an appropriate sub-forum.
Post Reply
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Find Duplicate Facts

Post by Kingy »

Mike

I have merged 2 trees (don't ask, it was a nightmare) but I have ended up with duplicate facts on some people.

Born in x duplicated
Census in xx duplicated
Occup. x duplicated
Died in x duplicated

But I see your plugin doesn't find these. Is there a reason you haven't and is it easy to amend for me to do if it can be done.

Thanks
Paul
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Re: Find Duplicate Facts

Post by Kingy »

Sorry just found reason myself, very subtle typing difference between the 2 entries.

Ignore will do it manually as I find them.

Thanks

Paul
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Find Duplicate Facts

Post by tatewise »

The only two items the plugin needs to identify duplicates is the Fact Name (obviously) and the Date.
Everything else can be disregarded by removing all the tick options.
So I guess the "very subtle typing difference between the 2 entries" is in the Date somewhere?

If it is something that other users could do by mistake, is it possible the Plugin could disregard those subtle differences and identify the Facts as duplicates to the benefit of all? Can you reveal what the subtle differences were?
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Re: Find Duplicate Facts

Post by Kingy »

I have attached an example which isn't being picked up.

Paul
Attachments
Capture.JPG
Capture.JPG (38.22 KiB) Viewed 1955 times
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Re: Find Duplicate Facts

Post by Kingy »

In the Census duplicate the Place is Town, County, England on one and just Town and County on other.

The occupation is exactly the same in both entries.

Paul
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Find Duplicate Facts

Post by tatewise »

Firstly, have you followed the Plugin advice and sorted all Facts into Date order? Tools > Re-order Out-of-Sequence Data...
Just because they are shown in Date order in the Facts tab means nothing.
The Facts tab always shows Facts in Date order regardless of database order.
To see what I mean, look at them in the All tab which is shown in database order.

Did you clear the tick options in the Plugin so that the Fact Place (PLAC) and other fields are disregarded?

The two Census facts on 30 Mar 1851 should get detected.

I can see possibly why the four Facts on 2 Apr 1871 are not detected if the two Census facts and the two Occupation facts are not adjacent.

The plugin works by detecting adjacent Facts with the same Label and the same Date.
So if the order is Census, Occupation, Census, Occupation then they are not spotted. I can investigate that rarety.
Almost any other order will result in at least one pair being detected and when resolved the other pair will be detected.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Re: Find Duplicate Facts

Post by Kingy »

Appreciate your help.

I wasn't aware of the fact the dates needed sorting via the sequence tool, which I ran and it found loads of out of sequence, so that helped, and I ran the find duplicate facts again and it found loads.

Strangely, the example I posted above wasn't one that changed. I understand what you are saying about not being next to each other, but that doesn't explain why the 1851 Census was unchanged.

I then moved the order manually to put the 1871 Census next to each other and the 1871 Occupation next to each other and ran again, and it found the duplicate occupation, but again not the Census.

Hope that is clear.

Paul
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Re: Find Duplicate Facts

Post by Kingy »

sorry forgot to add I did remove all ticks before running.

Paul
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Find Duplicate Facts

Post by tatewise »

Thank you. I've double-checked the Plugin script and discovered it does check both the Date and Place of adjacent Facts with the same Label, despite the Fact Place (PLAC) option being unticked. :oops:
I will look at that tomorrow and hopefully improve its ability to detect duplicates.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
User avatar
tatewise
Megastar
Posts: 28333
Joined: 25 May 2010 11:00
Family Historian: V7
Location: Torbay, Devon, UK
Contact:

Re: Find Duplicate Facts

Post by tatewise »

I've updated the plugin, and as a bonus, it now does not need Facts to be sorted, and should find any duplicates.

If all the tick options are removed then duplicates only need the fact Label, attribute Value, and fact Date to match.

Try attachment Find Duplicate Facts plugin Version 1.1.1 Date 05 Dec 2021.
Last edited by tatewise on 11 Dec 2021 16:02, edited 1 time in total.
Reason: Attachment deleted as Version 1.2 is in the Plugin Store.
Mike Tate ~ researching the Tate and Scott family history ~ tatewise ancestry
avatar
Kingy
Diamond
Posts: 62
Joined: 15 Dec 2020 10:33
Family Historian: V7
Location: Essex, UK

Re: Find Duplicate Facts

Post by Kingy »

Many Thanks Mike

That did the trick perfectly.

I really appreciate your help.

Paul
Post Reply