* Rich Text font detection

[Mark1834]

This short plugin arises from a wish list discussion on a proposed Plain Text Only setting, but is spun off separately to separate it from ongoing debate about the precise wording of the item.

It is a long-standing FH7 bug that it can introduce unwanted rich text formatting, usually but not always using the Microsoft system default Segoe UI font. While it is likely that CP will fix this bug in the future, that will not clear the (probably widespread) fault from existing projects.

This plugin is a first step in helping users to determine if their project is affected. Two options are available, to list all records and fields containing a defined font, or all records and fields containing any user-specified font.

Results are presented as a simple two column table. Double click on the first column to show the record containing the specified font. Double click on the second column to identify the specific field, which is highlighted in the opened Property Box (note that the rich text may be in a lumped source citation, as this is stored as part of the record).

The plugin is "read only" and makes no attempt to correct any issues it finds (so is safe to use on your live project). That would not be appropriate at this stage, as the first step should be CP fixing the original bug and enhancing the ability of FH to convert rich text back to unformatted text. If you have just a handful of affected records, they can be edited manually without too much effort. If the problem is widespread, it will probably be better to await a solution from CP.

The plugin is attached below. If the attachment has disappeared, there will be a later version further down the thread.

[LornaCraig]

Thanks for this Mark. It has revealed 15 cases of 'font contamination' in my project. Several of these were undetectable by eye because the main body of the text was in the correct default font. It turned out that there was a blank line immediately afterwards which was Segoe UI (usually) or Cambria Math (twice).

[jimlad68]

Just tested and works OK and I know this is a very early version.

Up to now (as previously mentioned), to produce a similar but "over comprehensive" list I used:

as part of my database "tidy up" routine I run the Plugin Search and replace (not the FH Find and Replace) select All records... , tick all basic filters, Search ONLY and look for <. This will also include legitimate < such as age less than, or instances where one may wish to use rich text.

- The ideal for me would be to list all "Rich text" (as Plugin Search and replace above does), but in categories such as Fonts, web links, individual links etc. That way it would be easier to spot legitimate from erroneous use of Rich Text.
- Next would be a click to change selected items to Unformatted Text. I am working on an AHK script to change a "text" note from Rich to Unformatted text, I am getting there but FH popout boxes and mouse controls are difficult to control, although my scripting skills are very limited and entail web searches and trial and error!

I am not against Rich Text, I have started to use it for web and Individual links, I just think it should be more transparent/ easy to monitor and control.

[Mark1834]

Enhanced detection by feature is already on my mental list for potential future refinement (and IMO is a reasonable thing for a plugin to do).

However, the fine control of edits that the wish list describes is difficult to achieve via plugins. They are much more suited to global changes, such as removing all formatting (globally, by record type, or by selected record), removing all font specification, etc.

Any plugin control of rich text should come after and complement the CP solution. As Helen pointed out in the other thread, while it could be achieved in days rather than months, a premature plugin sticking plaster would delay fixing the root cause.

[Jean001]

For me:

2112 rich text fields containing defined fonts selected

[BillH]

Mark,

A couple of questions...

When I run the plugin, and specify "Segoe UI" I get 80 results. When I specify "Segoe UI Symbol", I get 3 results. When I search for all defined fonts, I get 80 results.

My first question is why isn't all defined fonts giving 83 results instead of 80?

When I use Notepad++ and search for "Segoe UI", I get 117 matches. When I search for "Segoe UI Symbol", I get 39 matches. This is a total of 156 occurrences for both fonts.

My second question is why does the plugin show 80 and Notepad++ shows 156?

Thanks,
Bill

[Mark1834]

Good questions, Bill, thanks.

The plugin does a simple match for font name, so "Segoe UI" will return both plain "Segoe UI" and "Segoe UI Symbol", while "Segoe UI Symbol" only returns the full version. It could probably be enhanced in a later version with a more sophisticated pattern matching.

The plugin is counting the number of fields that have at least one occurrence of that font, while Notepad++ is counting individual occurrences so will in general produce a higher number.

[BillH]

Mark,

Thank you... that explains it.

I do think that refining the pattern match would be good. If I select "Segoe UI" out of the list, I would expect it to only find matches for that specific font, not any font beginning with those characters.

Thanks,
Bill

[Mark1834]

That was the intention, but clearly I didn't fully account for how the standard controls report the selected font!

[David2416]

Interesting: found 74 Segoe UI and 91 All Defined; 10,050 total rich text fields.

[AdrianBruce]

Oh that was interesting. I had very few - only one report was for a real Rich Text - a pasted report sourced from a Word document.

The rest were empty Segoe UI things. It did prove tricky to find some of the items - when double clicking on the 2nd column for these, I couldn't see any text of any sort. Eventually I twigged that in the case of a marriage event, the 2nd column was taking me to the marriage event but the marriage event note - hung immediately underneath the marriage event - was the one with the empty text.

(I dislike that pseudo-separation of event and note - IIRC, it doesn't happen for individual events but does for all family events.)

[tatewise]

Adrian, this is another case where the right-click context menu is useful.
Right-click Locate in Property Box is the same as double-click.
Right-click Locate in All tab displays the item in a suitable expanded All tab.
They are standard right-click context menu options in all Query and Plugin Result Sets.

You say:
I dislike that pseudo-separation of event and note - IIRC, it doesn't happen for individual events but does for all family events.
I don't understand. The Facts tab in the Individual Property Box shows all facts in exactly the same way whether Individual facts or Family facts. Where is the difference you describe?

Adrian has responded in the Fact Note Source Citations (21296) posting.

[jimlad68]

Not sure where to post this, but it seems like an "after the results" to the Plugin Rich Text font detection in the title of this post and until anything else comes along this is my workaround for:
In the Note/Text from source etc box popout from Property Box, convert existing Rich Text to Plain Text.

It is an AutoHotkey (AHK) script limited in particluar by: my ability, AHK mouse control, and FHv7 navigation/buttons etc.

The instructions are in the script, which also includes paste unformatted text with ctrl+shift+V.

From the Rich Text font detection plugin results, just doubleclick a "Rich Text" column result and run the script (check the script instructions first). As an alternative to Plugin Rich Text font detection, run the Plugin Search and replace (not the FH Find and Replace) select All records... , tick all basic filters, Search ONLY and look for <. This will also include legitimate < such as age less than, or instances where one may wish to use rich text. You can also add other search criteria, especially with LUA patterns, to hone in.

I have attached my AHK script for FHv7 in which this appears.

Code: Select all

; ============ Start section where these key combinations only work within FHv7 popout windows with ahk_class #32770.
; Note FH popouts are difficult to control.
#IfWinActive, ahk_class #32770

; -------------------------------------------------------------------------------
; In Text boxes - paste unformatted text
; Note popout box changes depending on paste source, hence the 6 x "p" which should always land on Paste Unformatted Text.
^+V::Send {Click, , Right}pppppp{Enter}    ;shortcut ctrl+shift+V

; -------------------------------------------------------------------------------
; 
; ACTION - In the Note/Text from source etc box popout from Property Box, 
;          convert existing Rich Text to Plain Text.
;
; THIS IS TO EMULATE:
; - In the Note/Text from source etc box
; - ctrl+A > ctrl+C > delete (Copy all the note to clipboard and delete original)
; - ctrl+alt+double click to edit the rich text
; - untick the Rich Text box > OK to exit rich text box;
; - Paste unformatted text, which should remove all the rich text features
; - Open Rich Text popout to check "Rich Text" tick box is NOT ticked.
;
; INSTRUCTIONS (once setup very quick)
; - Ensure the OK button in the "Rich Text Editor" popout via "ctrl+shift+double click" lies within the underlying text box.
;       (resize and/or drag as required - once there it should stay there for the FH session)
; - Ensure the Note/Text from source etc box is the ACTIVE window and the mouse position is within it.
;       (simply click mouse anywhere in the box and do not move the mouse)
; - Use the shortcut ctrl+shift+U
; - It should end up displaying the Rich Text popout titled "Note", 
;       you can check the "Rich Text" tick box is NOT ticked,
;       if it still is, see LIMITATIONS below.
; - Esc or Cancel to finish
;
; LIMITATIONS
; - certain "modern characters (I suspect not ANSI) are not converted to unformatted text
;          these need to be manually altered/deleted, then try again.
; - The "Rich Text" tick box is click ON/OFF so in some circumstances may end up ticked, 
;          so, just do manual click to finish.

^+U::   ;shortcut ctrl+shift+U
ControlFocus, RICHEDIT50W1 , ahk_class #32770
Send ^a^c{del}                                   ; - ctrl+A > ctrl+C > delete (Copy all the note to clipboard and delete original)
Send {LAlt Down}{LControl Down}{Click, , Left}{Click, , Left}{LAlt Up}{LControl Up}     ; ctrl+alt+double click to edit the rich text

ControlFocus, Button1 , ahk_class #32770        ; select Rich Text button
send {space}                                    ; space to flip tick on Rich text button NOTE IT FLIPS
send !O                                         ; select button OK
send +{F10}                                     ; R click menu

;       Note popout box changes depending on paste source, hence the 6 x "p" which should always land on Paste Unformatted Text.
Send {Click, , Right}pppppp{Enter}

Send {LAlt Down}{LControl Down}{Click, , Left}{Click, , Left}{LAlt Up}{LControl Up}  ; back to Rich Text popout to check "Rich Text" tick box is NOT ticked
Return

#IfWinActive  ;  ========== end of FHv7 ahk_class #32770 popout windows ===========================================

If anyone can improve it, please let me know, especially:
- the mouse control to move it to the active window, in this case the Note input area showing as ClassNN: RICHEDIT50W1 in AHK Window Spy, (in particlular in a multi monitor environment. AHK seems to be very erratic)
- A way of keyboard clicking the bar (with ...) on the RHS of the Note input area.

[Mark1834]

I have corrected the issue Bill raised, so selected fonts are now matched exactly, and also added further options to tabulate other common types of rich text field, such as tables, record links, etc. I experimented with including all rich text options, but the interface quickly became cluttered and confusing, so this simpler subset keeps it easy to use.

The updated main form is shown below. The initial font displayed is your currently selected Property Box default, as this plugin follows recommended FH7 best practice of using that display font such that plugin forms look very similar to how you prefer normal FH forms to look.

The attachment rich text.PNG is no longer available

Given the extended scope, a slight change of name from the first prototype is appropriate. This is now a general rich text tool rather than focusing exclusively on superfluous formatting, so my plan is to submit it to the Plugin Store in the new year once we have had a chance here to break it .

There is nothing here that conflicts with the recent Wish List item on plain text working, and it will remain a read-only tool for identifying what is in your project to aid manual review and audit, with no facility to make automatic changes to your data.

[BillH]

Mark,

It took me a few tries to figure out that the two columns were not combined. If I just want to find occurrences of Seqoe UI, I had to leave the options in the right column blank. Now that I figured that out, the plugin seems to be working just fine. I like the new options in the right column.

Thanks,
Bill

[Mark1834]

Thanks Bill. I’ll probably add a Help button to the Store version that goes into more detail about the options being “or” rather than “and”. It was potential confusion over whether selecting “Calibri”, “Bold” and “Italic” (for example) meant “Calibri Bold Italic font” or anything in Calibri, plus anything bold plus anything italic that led me to simplifying the options.

[BillH]

Mark,

My confusion wasn't in the font section but on the main window. If I selected Segoe UI and then selected Tables on the right, I thought maybe it would only show me tables that used Seqoe UI but instead it showed me all tables. Not a big deal. I now know that the font selection and the options on the right are not combined. If I want to look just for a specific font, I have to leave the options in the right column unselected.

Bill

[Mark1834]

The Rich Text Tabulation plugin is now available in the Plugin Store, along with a rudimentary help file. There are two minor enhancements over the prototype copy:

• The ability to search for any user-entered rich text tag.
• Menu selections are saved between sessions, making it much easier to repeat the same search.

I have also noted a sequence of steps that has a high probability of generating unwanted Rich Text, both on my desktop (FH 7.0.18) and laptop (FH 7.0.17.1) using a reset Sample Project, which does not contain any Rich Text fields. If you select all the text in either a Note or Text from Source field in the Property Box with <ctrl>a and start typing new text such that it overwrites the existing content, perhaps 20% of the time it is in Segoe UI font.

It's not consistent, but happens on two different PCs with separate versions of FH, and with multiple projects (not just the Sample Project).

[jimlad68]

The Rich Text Tabulation plugin is now available .......

Just tested this; observations:

[1] If you enter just < in Custom Tag it does not take. I notice other invalid Rich Text items e.g. bbb do not take. I was thinking that < would be a good catch all, BUT see [2] below. However I think an allowed search on < would be helpful.

[2] Selecting All Rich Text gave me many more items than by Search and Replace Plugin looking for <. This I tracked down to having NO rich text, but 2 _FMT 1 in the Gedcom. So that is an improvement for me over my previous method. Although these items should not be a problem when exporting or sharing the Gedcom as is.

[3] If not a big amendment, add a few more columns to the result, especially as this might help in sorting. e.g. Search and Replace offers Data ref and FH Search includes Type, Record Type and Record Id. I suppose there would be no end to it, e.g. BMD...

Many thanks for that, some nice enhancements and save me some more cumbersome checks.

[Mark1834]

Thanks for testing, Jim.

1. Yes, it searches for a single specified tag, so will accept only single line text starting with < and ending with >.
2. AFAIK, Search and Replace only checks the text content, but text can be defined as Rich Text even with no format codes, and this plugin checks the actual definition (corresponding to the presence of _FMT in the GEDCOM file).

[jimlad68]

Just tested custom with various, as you say it seems to need specific, so <> or <web> did not work, but <row> would.

[David2416]

Hi Mark,

I tested custom with this phrase <clr="000000"><hl="FF0080">#DNA </clr></hl> (without the bold emphasis).

It worked but on rerunning the Rich Text Tabulation screen had been resized with the following unfortunate effect.

Screenshot 2023-01-06 100823.jpg

It was easy enough to recover from this by editing the .ini file where the stored value was held.

[Mark1834]

Sorry about that, David. Plugin form design can be a pig to get just right when dealing with variable size elements while fully supporting user-defined fonts and zoom level.

My objective for plugin forms is that they should be indistinguishable from similar forms in the main program, and have a particular aversion to plugins that misbehave when resizing. That means I prefer fixed size forms where possible, but clearly I didn't get it quite right here!

The permanent solution is probably to do what I do with the Change Specific Fact Tag plugin, and permit horizontal expansion only. In the meantime, there is a simple workaround. Select Tool>Plugins>More>Edit to open the plugin editor, and change lines 149-150 to

Code: Select all

local dialog = iup.dialog{vboxForm; resize = 'YES', minbox = 'NO', maxbox = 'NO', border = 'NO',
	title = 'Rich Text Tabulation'}

That allows the window to be resized freely. Layout isn't quite right, but it makes it useable.

I'll post a revised plugin for testing later on.

[David2416]

Thanks Mark.

[Mark1834]

Actually there is a better solution now that I've had a little play...

Starting with the original plugin, add a new line after line 206, such that lines 206 and 207 now read

Code: Select all

	Options.Specified.Size = size
	Options.Custom.Size = size

This stops the frames expanding when presented with a long tag, so the tag wraps over multiple lines as necessary. Personally, I think this looks better than keeping it as one line with an over-wide form.

Optionally, you can also add an additional term to line 190 to read as follows

Code: Select all

	local tag = iup.GetText('Enter custom Rich Text tag', Options.Custom.Title or '')

This has the effect of copying the current tag into the definition box, making it easier to modify if necessary. On balance, I think this is better than always presenting a blank box.

These are both easy mods to make, so I will incorporate them into my master copy, but not submit it to the Store yet in case other issues arise as it gets more use.