* Problem with Paste Unformatted Text

[tatewise]

David, a quick experiment suggests your analysis is correct.

I set Tools > Preferences > Notes > Default Font to Segoe UI Symbol, 10 pt
Then pasted the Nicholas Hancher text from Bill's website as before using Paste Unformatted Text.
No font formatting was added and the \ escape was not before the < character, i.e. it was all plain text.

However, I then set Tools > Preferences > Notes > Default Font to Segoe UI, 10 pt
On repeating the same pasting, formatting for the Segoe UI Symbol font was added as when the Default Font is Tahoma.

So if it is known in advance which font the formatting needs then the formatting can be avoided.
Will FH be supplied with a crystal ball?

[Mark1834]

Agree, it's a bit too anaemic for my tastes. It might actually be the default font for legacy plugins that haven't been updated to respect the FH7 user-defined preference, so we may be using it already without realising .

[davidf]

So is Segoe UI available in FH7 Tools>Preferences>Display>Default Font?

• If it is do we eliminate a whole range of problems concerned with "unformatted paste" being unable to find a character in the "display default font"?
• Is the result of changing the FH Default Display Font to Segoe UI, tolerable - or does it look a mess compared to the Tahoma which most use?

Yes. IIRC it's present on all modern windows systems by default. And setting it as default seems to prevent the unexpected Segoe font switches. But (a) I for one hate it with a passion and don't want to work with it as my default font; (b) it wouldn't cure the Cambria Math and Segoe UI Symbol appearances, so it isn't a good workaround.

I set Tools > Preferences > Notes > Default Font to Segoe UI Symbol, 10 pt
Then pasted the Nicholas Hancher text from Bill's website as before using Paste Unformatted Text.
No font formatting was added and the \ escape was not before the < character, i.e. it was all plain text.

However, I then set Tools > Preferences > Notes > Default Font to Segoe UI, 10 pt
On repeating the same pasting, formatting for the Segoe UI Symbol font was added as when the Default Font is Tahoma.

So if it is known in advance which font the formatting needs then the formatting can be avoided.

So it would seem that Tahoma (assuming that is the font that is used by those having trouble) does not have a wide enough character set to avoid "non 'format-setting' font-switching").

Helen, for hates Segoe UI "with a passion" and it only offers a part solution (because it does not have a wide enough character set to address all instances of "non 'format-setting' font-switching".

1. Any "font expert" know of a visually acceptable font with a wider set of characters than either Tahoma or Segoe UI that might avoid this "non 'format-setting' font-switching"? (Hopefully a font that is widely available and not subject to license issues). That might give a work-around.
2. If "non 'format-setting' font-switching" is unacceptable, is there a consensus as to what we would prefer to have happen? Non switchable characters to be omitted or replaced with something like "□"?

[Mark1834]

Aren't we getting a little carried away with working a solution? I for one wouldn't find it acceptable to mess around with fonts to avoid these issues, so the onus is firmly on CP to fix it rather than devising elaborate workarounds.

[davidf]

Aren't we getting a little carried away with working a solution? I for one wouldn't find it acceptable to mess around with fonts to avoid these issues, so the onus is firmly on CP to fix it rather than devising elaborate workarounds.

That is my default approach, but

If the problem is shown to be "non format-setting font-switching" to be able to render a character in the clipboard (it's pointing that way, but do we have sufficient evidence - properly cited etc.! - to prove that conclusively?), there is little that CP can do but answer the two questions at the end of my post just above yours?

1. Any "font expert" know of a visually acceptable font with a wider set of characters than either Tahoma or Segoe UI that might avoid this "non 'format-setting' font-switching"? (Hopefully a font that is widely available and not subject to license issues). That might give a work-around.
2. If "non 'format-setting' font-switching" is unacceptable, is there a consensus as to what we would prefer to have happen? Non switchable characters to be omitted or replaced with something like "□"?

[ColeValleyGirl]

I use Calibri as my default font, which also doesn't appear to have a wide enough set of characters... Anyway, like Mark, I don't think we should be messing around changing fonts.

the onus is firmly on CP to fix it rather than devising elaborate workarounds.

And on Nick W as well of course as the same problem exists in AS... (sorry, Nick )

I very much doubt that Microsoft will modify the behaviour of their control, so we have to hope there's a configuration workaround.

[Mark1834]

It's worth noting as well that while Bill's example is good evidence for the experts to follow up, it is not the only cause of font contamination. I am 100% certain that it also occurs just when dealing with plain ASCII standard characters.

I haven't yet managed to determine precisely the circumstances to generate a reproducible example, and I treat it as an occasional annoyance that I deal with as I spot it (but clearly missed quite a few examples when I searched the GEDCOM ). Hopefully this enhanced level of attention will help CP understand what's going on.

[ColeValleyGirl]

Being a contrarian, I want to ask:

What are the real negative impacts of this 'font contamination'?

In a rich text note, I'd argue there's none (other than to display the pasted text correctly, which would seem to be more than a little desirable). Does it corrupt the other contents of the note in any way?

In what's intended to be a plain text note, it also enables the correct display of the pasted text but as a side effect turns the note into a rich text one. When viewed within FH is this an issue? On Export, the option already exists (in FH and in Mikes Export plugin) to convert Rich Text to Plain Text (and we should probably be selecting this option by default if we're exporting.) So why is the behaviour undesirable?

Are we seeing unexpected behaviour and assuming it's a problem? Is it really breaking anything? Or would documenting it for the people poking around under the covers be sufficient to set minds at rest?

[NickWalker]

And on Nick W as well of course as the same problem exists in AS... (sorry, Nick )

Not really because AS uses a normal text box if you want to use plain text and a rich-text box if you want to use rich-text. So AS already implemented the thing that has been requested for FH to do in the wish-list discussion.

[ColeValleyGirl]

Nick, fair enough -- but people poking around under the covers of a rich text note are still likely to see unexpected formatting.

FH already has a plain text editing option as well, it just isn't documented.

[NickWalker]

I haven't had time to read the article Helen found (well done for finding it) but as I said last night I suspect this will be a very difficult issue to solve, other than to give people the option of a 'plain text only' text box. If the note is implemented using a normal multi-line textbox (as seen in FH6 and earlier) you won't get this issue. It is because we're trying to display plain text in a rich-text box which obviously can't guarantee it will keep it plain.

I am making extensive use of rich-text for my sources so I'll expect my text to have formatting in it, so a few font changes to allow symbols to be displayed doesn't really make any difference to me. However, if you don't want to use rich-text then I can see this is annoying. Therefore I think the solution is to have a 'plain text only' mode as we've discussed which would just show a standard text box either on a note-by-note basis or as a global setting in FH. I do appreciate there is a hidden-mode to allow this now but this would need to be made much simpler and seamless for users.

[Mark1834]

And Nick is the only user to my knowledge to positively report no font contamination. Is that significant? Is AS intrinsically more reliable than the native FH font handling...?

As for whether it matters, that is purely subjective. Agree, it doesn't corrupt data, but it comes down to how tolerant you are of cosmetic appearance. If you rarely generate reports, you probably won't be bothered. If you like things to look neat and tidy and an app does something unexpected that compromises that, you get annoyed.

No right or wrong - but endless debate on subjective issues by retired folks on a cold winter's day is a FHUG speciality...

[NickWalker]

And Nick is the only user to my knowledge to positively report no font contamination. Is that significant? Is AS intrinsically more reliable than the native FH font handling...?

The only text I've added into FH7 note/source text boxes has been generated by Ancestral Sources. Ancestral Sources doesn't use any strange symbols that require these font changes. I'm not in the habit of pasting any text into my source text in AS as I use the auto-text feature. So I think that's why it hasn't become 'contaminated'. I'm not claiming any superior advantage here, it's just the way it works. But users of AS who only want to use plain-text would not have the 'rich text' option ticked and so they'd only see a normal text box and so any text they paste in would be guaranteed to be unformatted without any font changes.

[ColeValleyGirl]

As for whether it matters, that is purely subjective. Agree, it doesn't corrupt data, but it comes down to how tolerant you are of cosmetic appearance. If you rarely generate reports, you probably won't be bothered. If you like things to look neat and tidy and an app does something unexpected that compromises that, you get annoyed.

If it's corrupting the appearance of a note, I'd say that matters. Does it, other than the characters for which the font is switched? And are those characters always symbols of some type, of does it affect alpha-numerics (I'd guess not).

[Mark1834]

And are those characters always symbols of some type, of does it affect alpha-numerics (I'd guess not).

I've lost count of the number of times that I've said it also happens with plain boring ASCII text. We're going round in circles again, so that's my last word on this issue apart from supporting the recent plugin.

[davidf]

AS uses a normal text box if you want to use plain text and a rich-text box if you want to use rich-text. So AS already implemented the thing that has been requested for FH to do in the wish-list discussion.

So what happens when you try to paste text into a "normal text box" which included characters that are not in the existing font character set?

Options surely are limited:

1. Omit the characters - which can not just mess up appearance but potentially change meaning without the user realising?
2. Replace the "un-fontable" characters with a character like a square - which does at least alert the user to something on going on (or having gone on)
3. Use the nearest visual approximation in the font to the "un-fontable" character - how?
4. Use font switching to render the character as faithfully as possible - No per NW

[ColeValleyGirl]

Sorry, Mark, I must have missed that. Although FH doesn't use ASCII I understand what you mean.

[davidf]

And are those characters always symbols of some type, of does it affect alpha-numerics (I'd guess not).

I've lost count of the number of times that I've said it also happens with plain boring ASCII text. We're going round in circles again, so that's my last word on this issue apart from supporting the recent plugin.

So we need an example - together with the source from which it was cut or copied? Is the source "visually ASCII characters" (i.e. alphanumeric+) in a source that is rendered in a complex foreign font and when the paste happens, it is not trying to render "an ASCII character" but a "foreign" font's version of that character and the fall-back mechanism is using font-switching to sort out its confusion?

1. If it happens with plain boring ASCII text: that has to be a bug in the FH editor;
2. If it is alphanumeric+ text in a complex "foreign" font, the font-switching is failing to realise that it can render those characters in the FH-chosen font: is that a bug in the font-binding described in the article that Helen referenced (relatively recently, but so many posts ago)? That would be an OS limitation (The OS can look at what fonts an application like Office is using, but can't with non-MS applications like FH)?
3. The font-switching we have seen with "un-fontable" characters is a an OS issue workaround (presumably in the FH plain text editor?) which some don't like - in which case unless there is an alternative consensus to WL request, we live with it?

For an application accessed by keyboard short-cuts, that clipboard is damned clever, just not clever enough?

[tatewise]

The particular symbols in Bill's original website that gain Segoe UI Symbol font formatting are available in all the font sets that I have toyed with, including Tahoma, Times New Roman, Calibri, etc.

In the Rich Text editor window with those font formatting codes around just those symbols try some copy and pasting.
It is quite easy to accidentally upset the open and close font format codes such that other characters become formatted in the Segoe UI Symbol font too.
That is sometimes visible in the Rich Text editor window because the characters are rendered slightly differently.
It can also have an impact in Reports with an inconsistent rendering of otherwise identical characters.

So in answer to does it matter that some characters have unwanted and initially invisible font formatting then I would say yes.

[Jean001]

If it's corrupting the appearance of a note, I'd say that matters. Does it, other than the characters for which the font is switched? And are those characters always symbols of some type, of does it affect alpha-numerics (I'd guess not).

Yes, the appearance of a note is corrupted.

I (now) am aware that certain characters that I have used for years are seemingly problematic in v7 (I upgraded only in June).

In other software I regularly set a character/characters in a font different to the default font. That setting does not (nor should it) affect the rest of the sentence/paragraph/etc.

What I am finding, and that which I reported at the end of last month to CP, is that the font is changed for other text. Also, extra blank lines are being added at the end of a note.

I have also noted (notified to CP last month) that deleting the contents of a box does not delete all the unwanted font information (default settings are showing) but text entered into the (apparently) empty window is set in the unwanted font.

I have a Note Record (untouched by me for over a year) where I 'stored' useful characters (symbols). Since upgrading I see that some of the characters/symbols now have been set in Segoe UI Symbol or Cambria Math.

For years, since learning of the 'Configure Language and Accent Characters', I have stored my most-used characters there. They are affected by this problem. Likewise, characters copied from the Character Map.

I have used FH since about 2016, certainly 'actively' since then and, until v7 this June, I never had a problem in entering and using my most-used and other characters.

My ticket to CP (closed) is 'under investigation' by them.

[davidf]

The particular symbols in Bill's original website that gain Segoe UI Symbol font formatting are available in all the font sets that I have toyed with, including Tahoma, Times New Roman, Calibri, etc.

In Bill's website, the text in question is wrapped in <pre> tags (a bit like the code tags on this forum). He does not set any style for <pre> tags, so it takes the browser defaults. W3docs.org (which I have always treated as authoritative in matters HTML/CSS) states:

Text in a <pre> element is shown in a fixed-width font.

The tag content is displayed in the browser in a monospace font.

So in this case what ends up on the clipboard will be in your specific browser's default monospace font. So for me (Firefox on Ubuntu) that will be:

Poster's Font settings in Firefox (Edit> Settings>Fonts>Advanced)

Screenshot from 2022-12-14 13-50-58.png (40.85 KiB) Viewed 582 times

Others presumably can work out what is in their specific clipboard "paste" buffer by looking at their browser's monospace font.

Trying to paste "that" in "unformatted" seems to trigger font-switching. FH wants the characters but not in the monospace font - they are not "in plain old boring ASCII" but in something that the plain text editor cannot render. Presumably the font-switching happens in the font-binding in the paste functionality in FH (which I imagine is OS defined via some library of other).

[ColeValleyGirl]

Yes, the appearance of a note is corrupted.

Thanks, Jean -- definitely a problem then! I'll take my contrarian hat off...

[Jean001]

Yes, the appearance of a note is corrupted.

Thanks, Jean -- definitely a problem then! I'll take my contrarian hat off...

When I first noted the problems I thought it was poor typing by me (not for long though!).

[NickWalker]

AS uses a normal text box if you want to use plain text and a rich-text box if you want to use rich-text. So AS already implemented the thing that has been requested for FH to do in the wish-list discussion.

So what happens when you try to paste text into a "normal text box" which included characters that are not in the existing font character set?

With the character I tried it with last night, it looked OK to me in the normal text box and it looked fine in the raw GEDCOM file. The text uses unicode so has a large number of characters available to it. When I pasted it into the rich text box it added the font reference. The simplest answer to your question is that AS in plain text mode will behave in exactly the same way that FH 6/5/4/3/2/1 does when you paste text in.

[tatewise]

Trying to paste "that" in "unformatted" seems to trigger font-switching. FH wants the characters but not in the monospace font - they are not "in plain old boring ASCII" but in something that the plain text editor cannot render. Presumably the font-switching happens in the font-binding in the paste functionality in FH (which I imagine is OS defined via some library of other).

But those characters can be rendered in the plain text editor ~ as I've said several times.
( Simply remove the font-switching codes in the Ctrl+Alt [...] plain text editor and they render just fine. )
Furthermore, the font-switching chooses 'Segoe UI Symbol' font which is not a monospace font.

The default Tahoma font supports about 3,700 Unicode characters and Segoe UI supports about 3,900 characters.