* [Wish List 581] DOCX and/or ODT Report Generation Capability (was Hyperlinked footnote references in reports)

For Wish List Requests that have either (a) been progressed to the Wish List; or (b) been classified as duplicates, or as redundant because the requirement is already satisfied within FH and/or plugins; or (c) closed because it wasn't possible to arrive at a clear specification of the request within 15 months of it being raised.
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

AdrianBruce wrote: 18 Jan 2024 14:40 Do we need to add a requirement that a report saved in DOCX / ODT / whatever should look like the native FH Report? Currently, the PDF preserves the positioning of images when compared to the native FH Report, but the images in the RTF bear no relationship to their positioning in the native FH Report.

The Issue section currently says
The standard RTF documents support formatting in a limited sense, but lack any ability to incorporate graphics
My .JPG media do appear in the .RTF documents - just in the wrong place entirely. Or am I missing something?
Adian;
Currently FH7 reports do not have graphics, but may in future, so this would be an augmented FH7 capability that really has nothing to do with the reports. This aspect, however, is actually addressed in the issue statement as a justification for the requested change. All that said, a correctly implemented DOCX/ODT export capability inherently has the ability to contain graphics that faithfully represent the original. That is explicitly addressed by requiring adherence to the standard.
Gary Gauthier
Hunting History in the Wild!
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

My .JPG media do appear in the .RTF documents - just in the wrong place entirely. Or am I missing something?
Technically; standard RTF does not incorporate graphics. There is an augmented version called RTFD that does so. Some programs do support generation of that format variant. Perhaps your file follows that format? I chose not to get into that aspect, since most standards have variants that attempt to correct shortcomings. As you've discovered, some aren't a full solution. That is why I've emphasized DOCX/ODT that is fully compliant to the standard.

I hope it's OK to reference external sites in the forum. One might find the following article on Wikipedia to be of interest in describing RTFD and its shortcomings. [https://en.wikipedia.org/wiki/Rich_Text ... _Directory]
Last edited by Gary_G on 18 Jan 2024 16:19, edited 1 time in total.
Gary Gauthier
Hunting History in the Wild!
User avatar
ColeValleyGirl
Megastar
Posts: 5510
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Hyperlinked footnote references in reports

Post by ColeValleyGirl »

Gary_G wrote: 18 Jan 2024 14:18 It should also be noted that; while Markdown is also a commonly supported input for desktop publishers, it is far less standardized and its use therefore comes with higher compatibility risks.
I don't see the need to mention Markdown in the proposal; there is an item on the WishList to provide it already, and -- while it isn't ideal for desktop publishing -- it does have benefits for website publishing. Perhaps mention the Markdown request to explain why it's out of scope for this one?
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

ColeValleyGirl wrote: 18 Jan 2024 15:40
Gary_G wrote: 18 Jan 2024 14:18 It should also be noted that; while Markdown is also a commonly supported input for desktop publishers, it is far less standardized and its use therefore comes with higher compatibility risks.
I don't see the need to mention Markdown in the proposal; there is an item on the WishList to provide it already, and -- while it isn't ideal for desktop publishing -- it does have benefits for website publishing. Perhaps mention the Markdown request to explain why it's out of scope for this one?
Helen;
There have indeed been a number of posts regarding implementing Markdown as an export option from FH7. For me to fail to mention it here at all, and at least state why it is not truly a viable solution in this case, would invite requests to roll the noted requests into this wish-list item.

Markdown is also not as widely used as DOCX and/or ODT and employs quite a different paradigm for its implementation. To roll it in here would essentially be asking for CP to develop two different capabilities at once. I'll look at the wording and see if I can tweak it.

[I would like to mention that I have used it as part of a workflow for producing high-end print media. I've also used it for importing into Scrivener to produce some basic printed works. It has its place in print publication, but is markedly less common than DOCX or ODT. Unfortunately; most of the print-publication tools that use MarkDown really well have a workflow that is somewhat complicated.]
Gary Gauthier
Hunting History in the Wild!
User avatar
ColeValleyGirl
Megastar
Posts: 5510
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Hyperlinked footnote references in reports

Post by ColeValleyGirl »

Gary, I'm not suggesting you roll it in here, but rather note the existing request when explaining why it's out of scope.
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Second Draft

Changes in this revision:
  • Constraints section - Removed duplicated text in the following, "The implementation must use the standard MS default parameters, especially with respect to format tag-names tag-names."
"
------

Title
“DOCX and/or ODT Report Generation Capability”

Summary of Proposal
To facilitate the import and post-processing of Family Historian reports, the current output-format selection should be augmented to include DOCX and/or ODT.

Background
The final format and general appearance of a report is as much an artistic endeavour as it is a scholarly one. Individual expectations of what it can contain and how it is formatted is very much a personal preference. This may be as simple as including images or as complex as major structural changes and typesetting. Some programs supply extensive configuration parameters in an effort to meet user expectations. However; no autogenerated report can meet the needs of everyone and some form of post-processing by other applications is typically required.

Issue
As noted; end-users may need to post-process reports. To date, Family Historian has a limited set of standard output formats available; “PDF”, “Web Page (HTML)”, “Word-Processor Document (RTF)” and “Text File”. Of these; only the latter two are formats that are intended to be edited. The standard RTF documents support formatting in a limited sense, but lack any ability to incorporate graphics. Text documents can be edited, but provide even less formatting capability and cannot handle graphics or tables. Currently; even the more capable RTF files can barely produce output that closely resembles the appearance and content of the on-screen report. There are other standard formats, such as DOCX and ODT that can be used to generate output that closely mimics the appearance of on-screen reports. These two formats are also standard input formats for many word-processors and desktop publishers; the tools that are typically used for post-processing. In addition; should the report content of FH evolve to include graphics, then neither RTF nor TXT will be suitable. It should also be noted that; while Markdown is also a commonly supported input for desktop publishers, it is far less standardized and its use therefore comes with higher compatibility risks. In short; users would see significant current and future benefits from the addition of an DOCX and/or ODT export capability.

Proposal
It is suggested that FH7 be augmented to provide the ability to export reports in standard DOCX and/or ODT format.

Constraints
The following key requirements stem from observing ongoing difficulties encountered by FH competitors in implementing a similar capability.
  • To ensure reliability and compatibility with the largest number of word-processors and desktop publishers, the DOCX and/or ODT file must adhere to the respective published standards for the file format/content.
    [Note: Some programs, like Scrivener, may depend upon footnotes being implemented per the standard. This may require some care, since MSWord text-to-footnote linking works bidirectionally.]
  • The implementation must use the standard MS default parameters, especially with respect to format tag-names.
    [Note: Some programs, like Scrivener, depend upon the presence of default tag-names to mimic the document structure and correctly extract references upon import.]
  • Any optional customization and deprecated capabilities must be avoided in order to ensure the highest level of compatibility with third-party post-processors.
  • As the changes could easily result in regression of existing capabilities, it is recommended that the product be thoroughly regression-tested. The resulting output files must also be verified to be compatibile with commonly-used post-processors such as MSWord, LibreOffice and Scrivener (amongst others).
Scope of Changes
While, on the surface, the requirement is simply to produce reports in standard DOCX and/or ODT format, the necessary programming changes are likely to go well beyond the code responsible for generating reports. The target formats will likely require changes to other areas of the FH code to provide the required content to the report generator code.

This request has consistently used the phrase, “DOCX and/or ODT”, but it is hoped that both can be offered. While many post-processing applications accept both formats, some like Scrivener, seem to work best with ODT. However; if only one format is possible, then external converters between DOCX and ODT do exist.

To avoid confusion; it is intended that this wishlist capability, being focussed solely on the provision of a robust DOCX and/or ODT capability, is considered as separate and distinct from the following and any other Markdown-related wishlist items;
Last edited by Gary_G on 18 Jan 2024 19:44, edited 1 time in total.
Gary Gauthier
Hunting History in the Wild!
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

ColeValleyGirl wrote: 18 Jan 2024 16:42 Gary, I'm not suggesting you roll it in here, but rather note the existing request when explaining why it's out of scope.
Helen;
I trust that the second draft addresses your concerns.
Gary Gauthier
Hunting History in the Wild!
User avatar
AdrianBruce
Megastar
Posts: 2109
Joined: 09 Aug 2003 21:02
Family Historian: V7
Location: South Cheshire
Contact:

Re: Hyperlinked footnote references in reports

Post by AdrianBruce »

Gary_G wrote: 18 Jan 2024 15:01...
Adrian;
Currently FH7 reports do not have graphics, but may in future, so this would be an augmented FH7 capability that really has nothing to do with the reports. ...
Gary - are we at cross-purposes with respect to graphics in FH reports?

This is a screenshot of the start of an Individual Narrative Report on my grandpa.
Screenshot 2024-01-18 200445.jpg
Screenshot 2024-01-18 200445.jpg (116.15 KiB) Viewed 1185 times
The graphics that I'm talking about (which can be seen in that shot) are .JPG files that have been loaded into FH as Media Records and linked to my grandparents. The screenshot shows the Report as it is in FH, I can then Save Report As ... and save it as a .RTF. The images, however, appear in the wrong place in the .RTF compared to the native FH report. I am quite prepared to believe that the reason for the misplaced images is down to the limitations of the .RTF format but my point is that the starting point is that FH7 reports do currently have what I refer to as graphics...
Adrian
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Adrian;

Forgive me for having to revise this post so many times. I wanted to try and focus on what was giving you issues.

I think your concern may have to do with my saying, "The standard RTF documents support formatting in a limited sense, but lack any ability to incorporate graphics." Mea Culpa. I should have been more clear and said, "The standard RTF documents support formatting in a limited sense, but lack the ability of formats designed for word-processing and desktop publishing to incorporate and acurately position graphics". The point is that RTF is not capable of including the type of information needed to mimic what one sees in a PDF while DOCX and ODT are designed to do so. I will replace the sentence in the third draft of the wish-list.
Gary Gauthier
Hunting History in the Wild!
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Adrian;

I've given your concerns a bit more thought and I think that my rewording the Background section might make things a bit more clear for everyone.

----

Background
As noted; end-users may need to post-process reports. To date, Family Historian has a limited set of standard output formats available; “PDF”, “Web Page (HTML)”, “Word-Processor Document (RTF)” and “Text File”. Of these; only the latter two are formats that are intended to be edited. Text documents can be edited, but provide very limited formatting capability and cannot handle graphics or tables. Standard RTF supports formatting in a limited sense, but lacks sufficient information content to accurately lay out and format graphics and text. DOCX and ODT, on the other hand, were designed with word-processing and desktop publishing in mind and have no such limitation. So; DOCX and/or ODT are able to transfer a more “true” representation of the original report to a 3rd party program for further editing.

While not a currently supported output format in FH7, Markdown is a supported input to some desktop publishers and web-design applications. However; it would be difficult to cause it to transfer a "true" representation of an existing FH7 report. As such; it is not suitable for use in exporting pre-existing reports for the purpose of post-processing.
Gary Gauthier
Hunting History in the Wild!
User avatar
AdrianBruce
Megastar
Posts: 2109
Joined: 09 Aug 2003 21:02
Family Historian: V7
Location: South Cheshire
Contact:

Re: Hyperlinked footnote references in reports

Post by AdrianBruce »

Gary_G wrote: 19 Jan 2024 00:58... "The standard RTF documents support formatting in a limited sense, but lack the ability of formats designed for word-processing and desktop publishing to incorporate and accurately position graphics". ...
Thanks Gary - that makes sense now, both to justify your proposal and to explain why Calico Pie's RTF reports are so poor in relation to the positioning of the images.
Adrian
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Adrian;

Excellent!

Can I assume that you are OK with the text of the revised background section, in general?
It enlarges on what I said, just a bit, as text can also suffer the same fate as graphics.
If so; I will incorporate fit into Draft Three.

I will wait 2-3 days to see if there are more comments before posting that draft, unless I see comments that indicate that more are unlikely.
Gary Gauthier
Hunting History in the Wild!
User avatar
ColeValleyGirl
Megastar
Posts: 5510
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Hyperlinked footnote references in reports

Post by ColeValleyGirl »

Re
Gary_G wrote: 18 Jan 2024 19:35
You don't need to exclude this Wish List item as it will be replaced in its entirety by the enhanced text you provide (thus preserving the votes, and avoiding splitting future votes from people who find one request but not the other).
While not a currently supported output format in FH7, Markdown is a supported input to some desktop publishers and web-design applications. However; it would be difficult to cause it to transfer a "true" representation of an existing FH7 report. As such; it is not suitable for use in exporting pre-existing reports for the purpose of post-processing.
This may need a little rework -- Markdown is an eminently suitable format for some post-processing to produce e.g. websites (in fact it's easier in many respects than post-processing HTML output). Perhaps clarify to say it's not suitable for post-processing to produce reports that mimic an FH7 report? [Although, is mimicking an FH7 report essential? Or do people want to post-process reports to change the layout, for example?]

Otherwise, I think you're there.
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Helen;

1) As you asked to have me include such references, of course, I'll be happy to delete the one for "Save Report As options for extra file types (DOCX, ODT, etc)".

2) Wrt. the comment on the markdown paragraph, I've tried to address your concerns in the following;

Original:
"While not a currently supported output format in FH7, Markdown is a supported input to some desktop publishers and web-design applications. However; it would be difficult to cause it to transfer a "true" representation of an existing FH7 report. As such; it is not suitable for use in exporting pre-existing reports for the purpose of post-processing."

Suggested replacement for Draft Three:
"While not a currently supported output format in FH7, Markdown is a supported input to some desktop publishers and web-design applications. When exporting a document that must be an accurate reflection of the on-screen report, as in this case, it is not a suitable choice. This because, like RTF, it lacks the capability to accurately reflect the on-screen layout and formatting of report graphics and text. However; in instances in which such fidelity is not required, it may be a viable option. eg. as a basis for web-pages"

3) When one exports a document, especially when it is intended as the basis for further editing, there is usually an expectation that the unedited output file will allow the production of a close approximation of the viewed original. Adrian has amply has demonstrated this by his feedback on the shortcomings of RTF. The reason for wanting to then post-process a report is well explained in the background section.
Gary Gauthier
Hunting History in the Wild!
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Third Draft

Changes in this revision relative to Draft Two:
  • Issue section: rewritten to reflect comments by Adrian and Helen.
  • Scope section: to remove the reference to “Save Report As options for extra file types (DOCX, ODT, etc)”, as requested by Helen.
------

Title
“DOCX and/or ODT Report Generation Capability”

Summary of Proposal
To facilitate the import and post-processing of Family Historian reports, the current output-format selection should be augmented to include DOCX and/or ODT.

Background
The final format and general appearance of a report is as much an artistic endeavour as it is a scholarly one. Individual expectations of what it can contain and how it is formatted is very much a personal preference. This may be as simple as including images or as complex as major structural changes and typesetting. Some programs supply extensive configuration parameters in an effort to meet user expectations. However; no autogenerated report can meet the needs of everyone and some form of post-processing by other applications is typically required.

Issue
As noted; end-users may need to post-process reports. To date, Family Historian has a limited set of standard output formats available; “PDF”, “Web Page (HTML)”, “Word-Processor Document (RTF)” and “Text File”. Of these; only the latter two are formats that are intended to be edited. Text documents can be edited, but provide very limited formatting capability and cannot handle graphics or tables. Standard RTF supports formatting in a limited sense, but lacks sufficient information content to accurately lay out and format graphics and text. DOCX and ODT, on the other hand, were designed with word-processing and desktop publishing in mind and have no such limitation. So; DOCX and/or ODT are able to transfer a more “true” representation of the original report to a 3rd party program for further editing.

While not a currently supported output format in FH7, Markdown is a supported input to some desktop publishers and web-design applications. When exporting a document that must be an accurate reflection of the on-screen report, as in this case, it is not a suitable choice. This because, like RTF, it lacks the capability to accurately reflect the on-screen layout and formatting of report graphics and text. However; in instances in which such fidelity is not required, it may be a viable option. eg. as a basis for web-pages

Proposal
It is suggested that FH7 be augmented to provide the ability to export reports in standard DOCX and/or ODT format.

Constraints
The following key requirements stem from observing ongoing difficulties encountered by FH competitors in implementing a similar capability.
  • To ensure reliability and compatibility with the largest number of word-processors and desktop publishers, the DOCX and/or ODT file must adhere to the respective published standards for the file format/content.
    [Note: Some programs, like Scrivener, may depend upon footnotes being implemented per the standard. This may require some care, since MSWord text-to-footnote linking works bidirectionally.]
  • The implementation must use the standard MS default parameters, especially with respect to format tag-names. [Note: Some programs, like Scrivener, depend upon the presence of default tag-names to mimic the document structure and correctly extract references upon import.]
  • Any optional customization and deprecated capabilities must be avoided in order to ensure the highest level of compatibility with third-party post-processors.
  • As the changes could easily result in regression of existing capabilities, it is recommended that the product be thoroughly regression-tested. The resulting output files must also be verified to be compatibile with commonly-used post-processors such as MSWord, LibreOffice and Scrivener (amongst others).
Scope of Changes
While, on the surface, the requirement is simply to produce reports in standard DOCX and/or ODT format, the necessary programming changes are likely to go well beyond the code responsible for generating reports. The target formats will likely require changes to other areas of the FH code to provide the required content to the report generator code.

This request has consistently used the phrase, “DOCX and/or ODT”, but it is hoped that both can be offered. While many post-processing applications accept both formats, some like Scrivener, seem to work best with ODT. However; if only one format is possible, then external converters between DOCX and ODT do exist.

To avoid confusion; it is intended that this wishlist capability, being focussed solely on the provision of a robust DOCX and/or ODT capability, is considered as separate and distinct from the following and any other Markdown-related wishlist items;
Last edited by Gary_G on 19 Jan 2024 18:57, edited 1 time in total.
Gary Gauthier
Hunting History in the Wild!
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Helen and Adrian;

Had a few issues with cutting and pasting.
The Third Draft "should" now address your concerns.
Gary Gauthier
Hunting History in the Wild!
User avatar
ColeValleyGirl
Megastar
Posts: 5510
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Hyperlinked footnote references in reports

Post by ColeValleyGirl »

I'm happy, Gary.
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Thanks, Helen.

If Adrian can respond to indicate he is also happy, what is the next step?
Gary Gauthier
Hunting History in the Wild!
User avatar
ColeValleyGirl
Megastar
Posts: 5510
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Hyperlinked footnote references in reports

Post by ColeValleyGirl »

We wait a few days for any late input and (assuming nobody objects) then I go through the laborious process of turning your formatted text into raw html to update the wish list entry.

I wonder if i can persuade chatgpt to write me some python code to convert markdown bb markup to html :)
avatar
Gary_G
Megastar
Posts: 721
Joined: 24 Mar 2023 19:05
Family Historian: V7

Re: Hyperlinked footnote references in reports

Post by Gary_G »

Thanks, Helen.
I'll leave it with you.
Gary Gauthier
Hunting History in the Wild!
User avatar
AdrianBruce
Megastar
Posts: 2109
Joined: 09 Aug 2003 21:02
Family Historian: V7
Location: South Cheshire
Contact:

Re: Hyperlinked footnote references in reports

Post by AdrianBruce »

I'm also happy, thanks
Adrian
User avatar
ColeValleyGirl
Megastar
Posts: 5510
Joined: 28 Dec 2005 22:02
Family Historian: V7
Location: Cirencester, Gloucestershire
Contact:

Re: Hyperlinked footnote references in reports

Post by ColeValleyGirl »

Wish list Item 581 has been updated -- please vote for it if you would like to see this facility.
Post Reply