* Finding and Deleting duplicated weblinks with Search and Replace?

[MFriend]

Hi Folks:
I've been working on fixing up my wifes data on her side of the family. One issue I have is after importing from RM 8, I have weblinks that are duplicated on almost all citations (which is about 56,000). Here is what I'm talking about:



I figure there is a way using the search and replace plugin to delete the duplicate link. I know how limit the search and replace to just that part of the citation, I just don't know how to configure Search and replace to select and delete the duplicate link (assuming that is possible which it seems almost anything is possible with FH 7 or FH7 plugins).

Matthew

(P.S. this is not a problem caused by the import process. The data came from FTM 2019 and was exported to RM 8. Somehow along the way the weblinks in FTM 2019 were saved as both a weblink and a note in RM 8. FH7 when importing seems to combine the two resulting in a duplicate in that section)

[tatewise]

The Search and Replace plugin can be quite clever but detecting duplicated data is beyond its capabilities.
There are several published plugins that detect duplicates of various entities but none will find your duplicate web links.

A custom plugin will need to be written to perform the detect and delete process.
Are the duplicates always in a Web Links citation-specific metafield and are such metafields always of Type URL?

[MFriend]

Hi Mike:
Yes, they are always Weblinks and always type URL

Matthew

[tatewise]

On looking at this more closely, I think I have misunderstood the Web Links field details.
I don't think they are Citation-specific Source Template metafields at all.

I suspect you have customized the Citation Window so that it displays Web Links: on the Citation tab.
Furthermore, I suspect that is actually the local Citation Note field with Data Reference: SOUR.NOTE2
Please confirm, preferably with a screenshot of the Web Links: customization Edit Item window.

[MFriend]

Hi Mike:
I hadn't noticed that section had been changed also. In RM8 I only used the default generic templates to avoid issues when moving to online trees, etc. with sources and citations.

FH 7 during the direct import I guess makes some changes from the default.

Here is a pic of that field:


thank you,
Matthew

Unrelated I think, but even though I didn't use any special templates in RM8, it still imported the citation specific "Where within sourc" into the <Template Fields> section where is says ".Page: " I think the direct import though did do a good job of ensuring all the data made it into the new FH7 database without losing data.

[tatewise]

It seems that the RM8 import process has customized the Citation Window as follows:
Source Record
Title: no change
Template: no change
• Footnote: '<Template Fields>' definition from Source Template
• Short Footnote: ditto
• Bibliography: ditto
Source Text: 'Text from Source' definition with a new label (SOUR.TEXT generic field)
Citation-specific Details
• Page: 'Citation Details: <Template Fields>' definition from Source Template
Citation: 'Citation Details: Where Within Source' definition with a new label (SOUR.TEXT generic field)
Citation Text: 'Citation Details: Text from Source' definition with a new label (SOUR.DATA.TEXT generic field)
Web Links: 'Citation Details: Note' definition with a new label (SOUR.NOTE2 generic field)

I have written a custom plugin to handle duplicate Web Links that offers to list or delete the duplicate.
Try the attached Find Duplicate Web Links plugin Version 0.1 Date 31 May 2022.
It would be wise to create a backup before running the plugin.
Afterwards, the Edit > Undo Plugin Updates command will reverse its effect but only before closing FH.

[MFriend]

Thank you Mike, that worked perfect.

You saved me having to manually delete thousands of dupes

Matthew