* Why sources being listed more than once

[BillH]

I just produced a Descendants by Generation report and noticed something I had not noticed before.  

In the sources section of the report, one source is being listed 9 times.  In this case the same source is shown as source 12, 42, 106, 121, 136, 254, 269, 362, and 461.  The source takes up about 4 pages so all together that makes 36 pages when 4 would do.  It is doing this same thing for many of my sources.

Is there any way to prevent FH from showing the same source multiple times?  

Thanks,

Bill

ID:6821

[Jane]

I suspect the citations are slightly different, check your source settings tab as you may be able to improve the combining.

[tatewise]

This only arises when more than one Individual Record is selected to produce any Report.

If the Report Options on the Sources tab chooses At end of record on same page/a new page, then the Sources are created per Individual Record.
This can result in duplicates of the same Source, up to as many as there are Individual Records.
What else can the Report writer do?

Instead choose the At the end of the report option, then only one consolidated set of Sources is created without duplicates.

[LornaCraig]

tatewise said:
This only arises when more than one Individual Record is selected to produce any Report.

But as Jane said, it can arise if the details of the citations are slightly different, even when only one Individual Record is selected.

In Report Options > Sources tab, you can select 'Combine Identical Citations for same Source'. But there is also a section for 'Source Citation Information to Include'. If you have any of the items in this section ticked and the details differ for various citations (for example a page number in 'Where within Source') the citations will be treated as different and will not be combined.

[BillH]

Jane, here are the settings I'm using:



I'll check the citations more closely. I know the source is the same in each case, but there might be something slightly different about the citations, but I don't know what it could be.

Mike, I am only selecting a single individual. So, there must be something different that I'm just not seeing.

Thanks,

Bill

[LornaCraig]

Bill, it looks as if our posts arrived simultaneously. In view of what you say, do have a look at the 'Source Citation information to include' section, as that might hold the key.

[tatewise]

Yes, Lorna is spot on.
I have duplicate Source references where the Citations have different Where within Source or Text from Source or Notes, etc.
By unticking those fields in the Source Citation Information to Include options the duplicates are eliminated.

[BillH]

Lorna and Mike, thanks for the tip. That is exactly what was happening. I was including some of these pieces of source information and that was the difference. When I was comparing my citations I didn't notice because they are added on to the end of the citation without any blank lines before or after them which made it hard to see.

Is there some way to add a blank line between the source name, the source note, the text within source, etc?

Thanks,

Bill

[tatewise]

The only way to add line spacing is to add it to the fields themselves.
Although this is only possible for 'large text' fields with their own [...] edit box.

Adding blank lines at the beginning of 'large text' fields is easy - just type them in and they are retained.
But fields with labels (like Text From Source:) get separated from their label.

However, adding lines at the end of 'large text' fields is another matter.
A simple new line is not enough, it has to contain at least one character, otherwise it's automatically discarded.
The Text From Source field in the Source Record is happy with a new line and one space or tab character, which will space it from whatever comes next.
However, the Note field in the Source Record needs a new line and a printing character, which will be automatically followed by a full-stop.

FH is annoyingly inconsistent here!

[BillH]

Mike,

That is what I thought, but of course it isn't what I wanted to hear.  [smile]  We really should have more control over the format of the report.  Some simple blank lines between pieces of information would sure make the reports easier to read.

I have so many citation notes with information in them, the thought of having to go in and change them all manually is more than I think I want to try.

Would it take you long to write a plug in that would add a blank line to the front of every citation note? I hate to ask, but I know I wouldn't be able to write one.

Thanks,

Bill

[tatewise]

Here is the Plugin code you requested.

It assumes you have the Sources tab default Installation Settings for the Source Citation Information to Include i.e. all five options ticked.

This is important because if any of those fields exist, then the Citation Note already starts on a new line.
It is only if none of them exist that a blank line is needed at the front of the Citation Note.

If any of those options are NOT ticked then remove the associate Tag from the for intTag, strTag in ipairs({'~.PAGE','~.QUAY','~.DATA'}) do line.

If you want a blank line before the Note then insert 'n'.. in front of strNew..strRef near end.

With your large database it may run for about 20 secs, and needs running every time you make significant changes to Citations.

Code: Select all

   Code removed by Edit - MBT

[RogerF]

Just an admin question. Mike's post is timed at (03/10/13 - 15:56:09). That looks like an American date, with the time one hour in the future?

[tatewise]

On my system the date is correctly 10/03/13.
It only displays 03/10/13 while editing postings.
Certainly the time is an hour ahead.
Perhaps this is because FHUG uses American servers.

[BillH]

Hi Mike,

First of all, thank you so much for doing this.  I can't believe how fast you are able to write these.  You obviously have a better programming mind than I do.[smile]

tatewise said:
It assumes you have the Sources tab default Installation Settings for the Source Citation Information to Include i.e. all five options ticked.

I do have all five ticked.

tatewise said:
This is important because if any of those fields exist, then the Citation Note already starts on a new line.
It is only if none of them exist that a blank line is needed at the front of the Citation Note.

The note does start on a new line, but there is no blank line.  Here is an example.  



I would like to have a blank line before the citation note in all cases, even if I have any of the other citation information.  Actually, I guess I'd really like to have a blank line before the Text From Source info and the citation note info something like this:

 citation... Cit. Date... Assessment...

 Text From Source...

 note...

tatewise said:
If you want a blank line before the Note then insert 'n'.. in front of strNew..strRef near end.

I'm a bit confused.  Are you saying the plugin will not put the blank line before the note unless I add this?

Thanks!

Bill

[tatewise]

Bill said:
I can't believe how fast you are able to write these.

Before I retired I was a professional software engineer, so writing Plugins is 2nd nature.

Bill said:
The note does start on a new line, but there is no blank line.

Yes, but if there is NO Where within Source, Entry Date, Assessment, nor Text From Source then the Note continues on the same line as the Source Title.
It is this that the Plugin currently resolves.

Bill said:
Are you saying the plugin will not put the blank line before the note unless I add this?

Correct. Resolving the above scenario guarantees every Note starts on a newline, so adding 'n'.. ensures there is another newline i.e. a blank line.

Bill said:
I'd really like to have a blank line before the Text From Source info

Unfortunately, the Text From Source: label is created by FH, so any blank line can only appear after this label and before the 'info'.
e.g.[pre]2.  'Source Title' ... Cit. Date: ... Assessment: Unreliable.
   Text From Source:

   sample text from source

   sample note
3.  'Next Title' ... Cit. Date: ...[/pre]Be aware that there is no blank line after the Note and before the 'Next Title'.
If you would like a trailing blank line then after the while strRef:sub(1,1) == 'n' do ... end loop insert:

Code: Select all

                        while strRef:sub(-1) == 'n' do
                              strRef = strRef:sub(1,-2)                        -- Remove any trailing newlines
                        end

and add ..'n' to end of strNew..'n'..strRef near end, i.e. ,strNew..'n'..strRef..'n')
Unlike other fields this trailing newline is retained and is effective.

Would you like a label such as Citation Note: prefixed on each Note?
If so then after the while strRef:sub(1,1) == 'n' do ... end loop insert:

Code: Select all

                        if strRef:sub(1,15) == 'Citation Note: ' then
                              strRef = strRef:sub(16)                        -- Remove any leading label
                        end

and change strNew..'n'..strRef to strNew..'nCitation Note: '..strRef near the end.

[BillH]

Mike,

OK... I see what you are saying about Text From Source and its label.

Just to clarify.  To get the blank line before the note, I added 'n'.. as you said, which gave me 'n'..strNew..strRef.

To add the blank line at the end, you say to add ..'n' to end of strNew..'n'..strRef near end, i.e. ,strNew..'n'..strRef..'n').

But I don't see strNew..'n'..strRef anywhere.  Am I just missing it?  Do you mean to put it at the end of 'n'..strNew..strRef) so it would look like this 'n'..strNew..strRef..'n')

Thanks,

Bill

[tatewise]

Yes.
Strictly speaking, the 'n'.. should go between strNew and strRef, but it is not important in this context, because strNew is either '' empty or 'n' newline, so together they are either one or two newline characters.
i.e.
,strNew..'n'..strRef..'n')

[BillH]

Hi Mike,

I created a plugin with just the change to add a blank line before the notes.

I ran this on a small test database and it seemed to work fine. However, when I ran it on my full database, it ran for over 9 minutes and when it finally got done it had added two blank lines at the beginning of every citation note. At first it ran so long I thought that maybe the program had just locked up or something as I couldn't click on anything in FH while the plugin was running.

Thanks,

Bill

[mezentia]

Once again, as the contributions to this thread prove, we desperatley need more control over the formatting of not just notes, but events and attributes, and printed reports as well. I despair at the tortuous eforts needed just to get blank lines before/after notes. Quite simply, we should not have to resort to this level of chicanery. I want to spend my time recording my family's history, and I don't want to have to resort to programming to get blank lines in my text, or re-applying formatting to narrative reports every time I make an update in FH. If I put a blank line at the front/end of a note it should not get stripped out.

Rant over [smile]

[tatewise]

Bill, first of all I should have mentioned that because this Plugin alters Citation Note fields, it will change the Updated Date on virtually every Individual & Family Record.
If you don't want this disruption, you will either have to Undo Plugin Updates, or roll back to an earlier Backup archive, after every run of the Plugin.
Hope this does not cause a problem.

The long run time is a bit surprising unless your 10K database has a much higher proportion of Citations than mine.
If necessary a Progress Bar can be added.
It is these refinements that extend the development time of Plugins.

On the small test database, ensure that a representative sample of Citations are tested, by mimicking your large database Citations.
i.e. with/without Entry Date, Assessment, Note, etc, etc.

At present the Plugin determines the blank lines required and rewrites every Citation Note even if it does not need changing.
I have a refinement of the Plugin that only writes the Citation Note if it needs changing, which may make it run faster, and won't affect the Updated Date after the first run.

We also need to decide what to do about Citations with no Note, because they have no blank line before the next Citation.

As David says, we should not have to jump through these hoops for minor Report layout changes.

[BillH]

Hi Mike,

I completely agree with David that we shouldn't have to go to this length to get a report to have blank lines. This is just a workaround for a problem which hopefully will be addressed someday in FH itself.

I'll have to think about the update date changing on everything. I noticed that after my test run of the plugin. That probably won't bother me.

I have 2092 sources with 56,041 citations. I'm not sure if that is unusual or not. A progress bar would probably be a good thing if it continues to take so long to run.

My smaller test file was one branch of my family so it was probably pretty representative.

I guess I am most concerned about getting two blank lines instead of one. Any idea why that would have happened?

Your proposed changes sound good if they will speed up the run time. Although I would like it if there were a blank line before every citation, at this point you don't need to worry about that unless it is easy to do. I'm not putting blank lines after the notes, only before them to help them stand out from the citation itself.

[tatewise]

I have 2092 sources with 56,041 citations.

That is more than 25 Citations per Source, which is much more that I have.

If the test file is representative, I am surprised it was OK, and yet that same branch in your master file was not.

I hope you use Run in the Plugin window, and not Go in the Plugin Editor, which is much slower.

The Plugin adds a 2nd blank line when there is none of:
 Citation ~ Entry Date
 Citation ~ Assessment
 Citation ~ Where within Source
 Citation ~ Text From Sources

The Report Options Sources tab must have the Installation Settings:



Here is the revised Plugin but still without Progress Bar:

Code: Select all

   Code removed by Edit - MBT

[BillH]

Mike,

That would explain the extra blank line.  Most of my citations don't have those four fields.  However, when I was checking my test file I see that I was checking citations which did have Text from Source or Where within Source entries.  Now that I know what causes double blank lines, I looked at other citations and I do see the double blank lines there as well.

My settings are the same as the default except I include Text from Source and Notes under Source Information to Include.  Would that make any difference?

I'm really thinking that I wouldn't want the double blank lines.  Is there a way to suppress one of them if the other four fields are missing from the citation?

I'm going to be away till the weekend, so I won't be able to test the plugin more until then.

Again, thanks for the help.

Bill

[tatewise]

Yes, that explains the difference, and your settings are like my usual settings, so I had already got a solution under way.

See the next posting for revised Plugin.

[tatewise]

I have had another go at this Plugin.
This version checks the settings in your '\Reports\Standard\Descendants By Generation.fhr' Report Options.
Change this path in the Plugin if you are using a Custom Report file.
It should also run somewhat faster.

Code: Select all

   Code removed by Edit - MBT