* Pasting into Text from Source/Autotext

[arthurk]

I've finally upgraded to v.7, and one of the reasons for this was the possibility it held out of adding tabulated data to Text from Source fields. With the Lumped Source Splitter plugin (thanks, Mark) I've converted my lumped census sources into separate ones for each household, and am now looking at populating the Text from Source fields with tabulated transcripts. (These fields are currently empty.)

Over many years I've been keeping my own census transcripts outside of FH, currently in .docx format, but the text is arranged in tab-separated columns and can be pasted into a spreadsheet. (I use LibreOffice for these purposes.) What I'd like to do, if possible, is get this information into the source without having to retype it all.

This is an example of part of one of my transcripts, for 1841 - there are also column headers higher up the document. ("B12" etc is a Custom ID, and the bold also indicates a direct ancestor, but as these aren't part of the original census document I'm likely to ditch them at some point):

Original docx transcript.jpg (24.5 KiB) Viewed 1872 times

Going via a spreadsheet version I've copied the grid of 6 rows x 5 columns and tried to paste it into the table in the default 1841 census Autotext, but that didn't work:

Paste into autotext.jpg (49.1 KiB) Viewed 1872 times

However, I can paste the same thing into a blank part of a Text from Source field, and that produces an acceptable table (column headers, place, address etc would be added in the final version):

Paste as table.jpg (31.6 KiB) Viewed 1872 times

So in a nutshell, is there a way to paste data into an existing table in Text from Source, or can you only do it by creating a new table?

Additional comments:
(a) I'm aware that the columns in FH's 1841 Autotext table don't match mine (or the original), and once I get going I think I'll probably create a custom Autotext table with Age (male) and Age (female) columns rather than the default Sex and Age ones (unless anyone can explain why the default one might be better).
(b) I've never used Ancestral Sources, nor have I explored Data Entry Assistants, but would consider them if they would help with what I'm trying to do; however, I don't need any new sources or facts, all I'm trying to do is add some text to existing sources.

Does anyone have any suggestions, please?

[tatewise]

I have not tried various methods of migrating data into rich text tables, but I suspect that it is only possible cell by cell or as a whole table into a blank area, as you have found.

One reason for combining male & female ages into one column is to minimise the table width.
If all the Census columns for occupation, place of birth, etc, are added then it can get quite wide.
In FH Reports, tables that are too wide are not adjusted to fit the page in any way and is a known problem.

If you intend to use Ancestral Sources then you may wish to make your Text from Source format to match its format.
Otherwise, you could end up with two different styles of Source record format.
The same argument applies to other types of Source and not just Census records.

[arthurk]

If you intend to use Ancestral Sources then you may wish to make your Text from Source format to match its format.
Otherwise, you could end up with two different styles of Source record format.
The same argument applies to other types of Source and not just Census records.

Thanks, Mike - though in some cases I suspect it might be more that I'd need to spend (quite a lot of) time getting Ancestral Sources to produce something that matches my existing sources.

Anyway, given that the Text from Source in my census sources is currently empty, the questions would be:
(a) can AS be used to edit existing sources, without creating new facts?
(b) if so, is it any better at handling pasted data than FH's built-in Autotext?

[tatewise]

Yes, customising AS to match your Source record format is an alternative approach.
You will need to edit every Census Template for every Country for every Year to have for example two Age columns.

(a) Yes, AS can be used to replace/supplement existing Source Citations without replacing existing Facts, but it does not edit existing Source Citations. The best advice is to experiment with a Copy of your Project to avoid damaging the master.

(b) No, AS is even less capable than FH and only one cell at a time can be pasted. AS is primarily designed to capture details off a newly acquired source document such as a Census Return, Birth Certificate, Marriage Certificate, Burial Parish Record, etc.

[NickWalker]

Although not necessarily relevant to this discussion, but responding to Mike's comments about AS... I have been working for a while on a new feature for AS which can be used to convert census source text that was recorded in plain text with earlier versions of Ancestral Sources or its predecessor Gedcom Census, into rich text tables. This is now working on my development version but needs some additional work before I can get to the stage where it is ready for testing. I'm also aiming to have the ability to convert the generic source into a templated version as part of the process for those who would wish to do that.

[ColeValleyGirl]

Although not necessarily relevant to this discussion, but responding to Mike's comments about AS... I have been working for a while on a new feature for AS which can be used to convert census source text that was recorded in plain text with earlier versions of Ancestral Sources or its predecessor Gedcom Census, into rich text tables. This is now working on my development version but needs some additional work before I can get to the stage where it is ready for testing. I'm also aiming to have the ability to convert the generic source into a templated version as part of the process for those who would wish to do that.

A bit like the Convert text from Source to Rich Text plugin?

[arthurk]

....
(a) Yes, AS can be used to replace/supplement existing Source Citations without replacing existing Facts, but it does not edit existing Source Citations.....

(b) No, AS is even less capable than FH and only one cell at a time can be pasted......

In view of that, then, I don't think I shall be using AS for this particular task.

I've had a slightly deeper look at the default Autotext, and the way tables are recorded in the Gedcom file, both the default ones and the ones I've pasted in, and a few thoughts are beginning to emerge:

1. I take the point about the way tables appear in reports, but having tried with both the default table and the narrower ones that I pasted in, I'm not sure that I'm going to want to include these census ones anyway.

2. Nevertheless, the default 1841 census table seems far wider than it needs to be, so if/when I produce my own custom version it will be a lot narrower, and would fit in a report better if I did ever want to use it. How much I'll be able to do this with later censuses remains to be seen.

3. Rather than just dragging column dividers and table edges and hoping for the best, it would be nice to be able to define the table and column widths precisely so as to achieve some consistency between eg 1851 and 1861 census tables. I've found the column width definitions in the Autotext files (and in the source record in a Gedcom file) as a series of numbers, but is there an easier way to access or define them? Even just a ruler across the top of the Note window would help.

4. In order to achieve this consistency, I'm beginning to wonder if the best option at present is to do the bulk of the work in a spreadsheet program (or word processor if there's anything outside the table) where column and table widths can be defined (and where I can also paste in my existing transcripts), and then paste the finished result into an empty Text from Source field.

Does anyone have any thoughts on this?

[Mark1834]

The question for me is “why do you want formatted tables?” IMO, there are too many columns in census returns to display neatly in a typical A4 portrait report.

I record census text as a simple column-separated literal transcription of what’s in the return, apart from a standard abbreviation of the County of birth (e.g. John Smith, Head, Mar, M, 34, Ag Lab, b Newtown CAM). Similarly for GRO Certificates and other wide documents in a fixed format.

It looks neat in reports, and easily located if I’m searching for a particular text string. Granted, it’s harder to extract the data in a structured way, but most of the key data are recorded separately anyway. You pays your money and takes your choice - whichever method best fits your priorities.

[arthurk]

The question for me is “why do you want formatted tables?” IMO, there are too many columns in census returns to display neatly in a typical A4 portrait report.

I record census text as a simple column-separated literal transcription of what’s in the return, apart from a standard abbreviation of the County of birth (e.g. John Smith, Head, Mar, M, 34, Ag Lab, b Newtown CAM). Similarly for GRO Certificates and other wide documents in a fixed format.

I take it you mean comma-separated?
Otherwise, as I doubt I'll be using these much in reports, getting them down to A4 size would be nice but not essential. The main thing for me is to have a kind of reproduction of the census entry (or certificate etc as well in due course), which in some cases would mean I wouldn't bother with an image.

With the way you do it, how do you handle empty fields (eg someone with no occupation)? Do you just ignore that field, or do you use 2x commas, with or without a space between? (I abhor strings of commas.)

Moving on from my own thoughts:

3. Rather than just dragging column dividers and table edges and hoping for the best, it would be nice to be able to define the table and column widths precisely so as to achieve some consistency between eg 1851 and 1861 census tables. I've found the column width definitions in the Autotext files (and in the source record in a Gedcom file) as a series of numbers, but is there an easier way to access or define them? Even just a ruler across the top of the Note window would help.

4. In order to achieve this consistency, I'm beginning to wonder if the best option at present is to do the bulk of the work in a spreadsheet program (or word processor if there's anything outside the table) where column and table widths can be defined (and where I can also paste in my existing transcripts), and then paste the finished result into an empty Text from Source field.

I've discovered that although the column width figures aren't easily visible, they are nevertheless found by the Search and Replace plugin. I think that means that if I were to put the sources for each census year into a separate named list, I could then take a year at a time and do a Lua pattern search for whatever numeric values there might be in the width definition string and replace them with those of my choosing, so as to at least get consistency across all the ones for a particular year. Am I right?

[Mark1834]

Yep, comma! I ignore blank columns, no extra commas. I am not interested in the precise formatting, so they add no value and look ugly. I always preface place of birth with ‘b’, and would add similar annotation to other fields if there was risk of ambiguity. For me, the transcription and image are BOTH required. The image is your original source, and a transcript merely your interpretation of it.

[NickWalker]

A bit like the Convert text from Source to Rich Text plugin?

Well vaguely, but that plugin couldn't cope with the way a lot of my census records are formatted and the whole process would be very laborious. AS will automatically fill in the autotext template (using whatever fonts, colours, titles, etc. the user currently uses), create hyperlinks to the individuals, etc. AS will also autofit the column widths (which FH doesn't do). I'd always planned to do this, even long before FH 7 started being tested, it's a big project to do correctly, but it's coming along. I first promised I'd do this 14 years ago (when AS was called Gedcom Census) as you can see from this post!

[NickWalker]

The question for me is “why do you want formatted tables?” IMO, there are too many columns in census returns to display neatly in a typical A4 portrait report.

I record census text as a simple column-separated literal transcription of what’s in the return, apart from a standard abbreviation of the County of birth (e.g. John Smith, Head, Mar, M, 34, Ag Lab, b Newtown CAM). Similarly for GRO Certificates and other wide documents in a fixed format.

It looks neat in reports, and easily located if I’m searching for a particular text string. Granted, it’s harder to extract the data in a structured way, but most of the key data are recorded separately anyway. You pays your money and takes your choice - whichever method best fits your priorities.

Comma separated tables have problems when there are commas in the data itself (I know this because this is the way I did it for years!). RTF tables look nicer on the screen (I don't print my family history out) and by having the data in a structured format it it is a very simple task to export the data into other formats (e.g. html) if I ever need to. For me I use the source text to be able to see the data at a glance in FH without having to read the image (although I have that too when required). If I ever did need to print them out I could split the tables onto different rows (and I would hope a future version of FH would do this automatically if required), shrink them into a smaller font, etc. AS does a good job of reducing the columns down to fit the data.

[arthurk]

After reading all the comments, I'm still leaning towards putting my census data in tables; I accept that these don't work well in printed reports, but as I don't use these much I think I can live with that. These are going to be mainly for my own on-screen benefit, and in view of what Nick says about the way he and AS handle tables ("autofit the column widths"), I'm now wondering if I've been overthinking a bit and shouldn't worry too much about having all of each year's tables exactly the same.

(Incidentally, in an earlier post I wondered if I could use the Search and Replace plugin in Lua pattern mode to adjust column widths for all the census sources for a particular year if I felt it necessary. No-one responded to that at the time - does it sound feasible?)

I think now I need to try a few things out and see which is easiest:

(a) using a spreadsheet, turn my existing census transcripts into tables and paste them into text from source

(b) retranscribe/retype the census data into FH's default autotext tables

With both the above, adjust table and column widths manually to fit the contents.

(c) retranscribe/retype with AS, creating new duplicate versions of my existing sources, then use the Replace Selected Source Citations plugin to switch them over

Thanks for all the thoughts.

[tatewise]

Sorry, yes, using the Search and Replace plugin with Source records selected via a Named List should work as you suggested.

[arthurk]

Thanks, Mike.