* Exclude citations from reports unless assessed as Primary

[Peter Collier]

Would anyone be willing to write a plugin to achieve the following? Unfortunately, I cannot code.

I would like to use the citation assessment field (which I otherwise do not use) to mark which citations should appear in reports and which should not. This is in order to limit the number of citations for each event/attribute, which can otherwise become excessive, so that only the most pertinent source(s) for each fact are referenced in any given report.

The plugin would need to hide or temporarily remove any citations assesed either below a user-nominated level or below "primary" (if the plugin has to be permanently set to a particular level), so that a report could then be produced without them. Perhaps the plugin could even prompt for the required report first, and then produce it automatically? There would need to be a way to restore the excluded citations once the report had been generated and saved.

If the plugin has to function in such a way that it actually removes the citations before subsequently restoring them, it might also be a good idea to have the plugin prompt for a backup before it fully executes.

Many, many thanks in adavance to anyone is is able and willing to help!

[tatewise]

Hi Peter, try the ATTACHMENTS Remove Citations Without Assessment prototype Plugin Version 0.1 Date 03 Mar 2016.

It has warnings at the start regarding Backup/Copy Project, and advice at the close on how to restore the Citations removed.

I advise that you use File > Backup/Restore > Small Backup before running Plugin just in case!!!

The Plugin currently removes every Citation that has no Assessment value.
I chose that method to avoid a user dialogue, and make it quicker to code.
If you decide that it is important to be able to select the Assessment cut-off level, then that can be added later.

Unfortunately Plugins cannot perform commands such as File > Backup and Publish > Report.

There are two methods to use the Plugin:

1. Create a Copy Project then after running the Plugin on the copy, and producing all required Reports, then the Copy Project can be Deleted and there is no risk to main Project.
2. Use File > Backup/Restore > Small Backup before running Plugin, produce all required Reports, then use Edit > Undo Plugin Updates before closing FH, or if you forget then restore the Small Backup.

[Peter Collier]

That was an unbelievably fast response. Thank you very much, Mike. Your efforts in helping everyone on here deserve a medal.

I've been running some tests on on a copy of my file and the plugin works very well - does exactly what it says on the tin. It could be left as it currently stands.

If you want to improve it further, then from my point of view there is only thing. Currently, the plugin only deletes the source if there is no assessment attribute at all, but if there is a blank assessment attribute it still leaves that source in place (i.e. the source has the attribute but the field value is null rather than primary, secondary, questionable, or unreliable). To my mind, it would be better if control were by a change in the value of the assessment, rather than actually adding or deleting the attribute outright.

It doesn't matter what value acts as the trigger, since it's a binary condition. Primary would be the most logical I suppose (if "Primary" then next else delete).

Given the plugin's current behaviour I'm wondering though if I could have my cake and eat it here too. If a null value were used as the trigger (if "" then next else delete), then it would be possible to use the assessment attribute properly in future if I wanted, and just delete the value on the sources I wanted to keep before running the plug in.

What do you think?

[tatewise]

The first thing to understand is that a field with no value is a transient state that only exists temporarily after its value is deleted. As soon as the Project is saved and reopened, the empty field is automatically deleted and no longer exists. It is this stable situation I was catering for.

I do not understand what you mean by "it would be better if control were by a change in the value of the assessment". In general the Plugin cannot detect a change in the value because it has no knowledge of history. The only temporary change of state it can detect is a deleted value as explained above, but I am not keen on using such a temporary condition.

I was working on the basis that you would have a static Assessment value for each Citation. Those that you wanted in Reports could have any Assessment value, and those without a value would be excluded. This is based on your original concept of having one or two 'best' Citation(s) and the rest as subsidiary and not required in Reports. They could still have a pecking order by sorting them in the Sources For pane.

I was not expecting that you would be wanting to 'fiddle' with the Assessment values just before running the Plugin and producing the Report. That sounds like a recipe for inconsistent results, and potential risk to the Project if you forget to Undo all the changes.

If you need further refinement to the Plugin to choose the Assessment cut-off value then that is easy to add. I just wanted to give you something to experiment with quickly. With a cut-off value then you can set an Assessment value on every Citation and choose to keep only highest values in Reports.

An alternative scheme is for the Plugin to only keep the first Citation for each Fact so only the 'best' Citation to date is reported. A slight variant would be to keep the first Citation and all subsequent ones with the same Assessment value for the same Fact.

[Peter Collier]

[edited]

Hi Mike,

I wasn't aware the empty field would be deleted when the project was next opened - I didn't think to check for that. You learn something every day!

You were working on the correct basis - you created what I had imagined. When I 'discovered' (as I wrongly thought) that the assessment field could have a null value, that sent my thinking off on a tangent as there was the potential for the assessment field to fulfil its original 4-value role (if I ever decided to use that in future), while having a null value as the switch for the plugin. However, since I was wrong and the field cannot actually be (permanently) null, then my mental wandering has led me down a dead end. Back to Plan A.

Sorry if I confused you with a poor choice of words. When I said a change in the value of the assessment, I didn't mean that the plugin should be aware of the assessment field's historical state. I meant only that I would prefer to have an assessment field exist on every fact by default and manually change the value as required, rather than having to delete/create the attribute outright. I think that may prove more reliable for me.

By reliable, I mean it would be better to have an extra citation on the report that I don't need displayed, than to not have one that I did. Since it might be easy to overlook a couple of citations among thousands if I first have to find and add assessments to them before publishing reports, I feel it would be better if I instead set up every citation with a default assessment value that will have the plugin keep the citation, and then change the ones I want to delete. That way the report will fail safe.

Could you refine the plugin, so that it will ignore (not delete) any citation with its assessment set to Primary, and delete all other citations (i.e. with assessment set to a value other than Primary, or with no assessment present)?

[tatewise]

We seem to have drifted away from the objectives in your original Floundering in a flood of footnotes (13488) posting.

What you originally wanted to do was avoid the preponderance of a long set of footnote numbers on any one Fact.

That is why I suggested keeping just the first and presumably 'best' Citation for each Fact regardless of its Assessment.

A refinement might be to keep subsequent Citations that have the same non-empty Assessment value as the first Citation.

I think on reflection that is better than having an Assessment cut-off value.

e.g. You could have a few equally valid Citations, each with an Assessment of Secondary, that a Primary cut-off would remove, leaving no Citations for that Fact. Whereas my latest idea would retain them, but any subsequent lower value Assessments would be removed. In effect what I have proposed is a floating dynamic cut-off value equal to the Assessment value of the first Citation against each Fact. This has the advantage of needing no user dialogue to set a cut-off threshold, and yet retains the 'best' Citations against every Fact, whilst automatically removing the lesser Citations.

[Peter Collier]

Yes, there has been a little mission creep. And people wonder why government projects run past deadline and over budget!

I see where you are going with your idea about matching the first and equally assessed subsequent citations. That might be a neat little trick. Thinking of the three or four thousand citations I have input already with no particular regard to their relative order, or of future occasions when I may come across a better source to cite than what I have used up until then, is there a way to easily rearrange the order of existing citations or do they have to be deleted then re-added in the desired order?

[tatewise]

I presume there is nothing in each Citation to automatically determine its order, such as Assessment values.

The manual method to alter the order of Citations is to use the up/down black arrows near top right corner of the Sources For yellow pane.

[tatewise]

Overnight I have given this a little more thought.
I asked myself "How would Calico Pie implement this feature?" and "Are they any precedents already in FH?"
The answer I came up with was an Exclude from Reports option as exists in Edit Media Item dialogues.

So my thinking now is that Calico Pie might be persuaded to add an Exclude from Reports tick option per Citation in the Sources For pane.

In the meantime, the Plugin could be easily modified to detect the text "Exclude from Reports" in the Note field of any Citation and hide those Citations.

The advantage of this technique is that you don't have to change any Citations or their Assessments or their order if you don't wish to. All you need to do is add the text "Exclude from Reports" in the Note field of the few Citations to be excluded from Reports.

BTW: I have also thought of a way of keeping the 'hidden' Citations linked to the Facts via a Source Note and minimise the risk of losing them.

[mjashby]

Mike,

To pick up on your thinking of how Calico Pie might look at implementing this feature, my own view is that the 'privatisation' of file content in FH is sadly lacking (or at least under-developed) in comparison with other competitor products. It should be relatively easy for the developer to include more privatisation features, e.g. a 'switch' in the Sources Entry Pane would allow individual sources and/or citations to be 'privatised' and then only a single option would need to be selected by users when creating reports or even gedcom output: "Include/Exclude privatised data".

Mervyn

[davidm_uk]

bbbbut..... I thought that the original requirement was about keeping reports tidy, where there were multiple citations for a particular Fact, and only wanted to show the "primary" one in the report (and maybe a given citation might be the "primary" one for some Facts/Individuals, but only "secondary" for other Facts/Individuals - I'm not sure if using the Note field would allow for this)?

Privatisation of Individuals, Facts, Citations, Notes, Places etc. etc. is a whole different kettle of fish, and much more complex to both implement and use.

[tatewise]

David, did you look at the original posting Floundering in a flood of footnotes (13488) that did not mention Assessment values at all and only wanted a way to control which Citations get Reported. It appears that the user does not currently use Assessment values at all, or even have the Citations in any sort of meaningful order.

I introduced the concept of using Assessment values as the 'switch' to decide which Citations to exclude. But on reflection a separate mechanism such as an Exclude from Reports 'switch' would be better. Mervyn, such a 'switch' would effectively perform the same function as a Privatise 'switch' if the Report Options > Sources tab had an override option. (In the meantime the Plugin under discussion does the same job.)

Until such a 'switch' is implemented by FH, my proposed workaround is to put "Exclude from Reports" in the Note field, which just like the Assessment and Where within Source fields is specific to each Citation and quite separate from the Source record. Then the Plugin would temporarily hide the Citation (not the Source record) while the Report is produced.

So to answer David's question, yes a single Source record can be cited by different Facts and have Assessment = Primary with an empty Note field in one, and Assessment = Secondary with a Note field containing "Exclude from Reports" in another.

[Peter Collier]

a given citation might be the "primary" one for some Facts/Individuals, but only "secondary" for other Facts/Individuals

You're right that that distinction does need to be made. However a citation is the bridge linking one fact to one source, so that problem doesn't arise. My original problem was that you can currently only privatise sources, which thereby knocks out all of its citations, including the ones you do want as well as the ones you don't.

[Peter Collier]

The advantage of this technique is that you don't have to change any Citations or their Assessments or their order if you don't wish to. All you need to do is add the text "Exclude from Reports" in the Note field of the few Citations to be excluded from Reports.

Mike, that sounds like a good idea. The current method works just fine for me, but hanging a note field off a citation instead avoids the need to re-purpose the assessment feature and so makes the plugin available to all.

[davidm_uk]

So to answer David's question, yes a single Source record can be cited by different Facts and have Assessment = Primary with an empty Note field in one, and Assessment = Secondary with a Note field containing "Exclude from Reports" in another.

Ah yes, after experimenting with the FH Sample Project I see now. I was confusing the Source Note with the Citation Note.

[tatewise]

Try the ATTACHMENTS Hide Citations to Exclude from Reports prototype Plugin Version 0.1 Date 05 Mar 2016.

This checks the local Note of each Citation to see if it contains Exclude from Reports anywhere.
If so, the Citation is moved to a local Source Note saying One or more Source Citations excluded from Report.

The Report will then include that Source Note in the Sources section, but there will only be one such entry per Fact regardless of how many Citations were excluded, and it is a one line entry instead of all the usual Source fields. So even though hidden or privatised they are not entirely forgotten.

So where there was a long list of Citations there could be just one Primary Source Citation and one Source Note hiding all the other Citations that have Exclude from Reports in their Note field. That applies to both the Property Box and the Reports for any Individual or Family. I have assumed that Source Citations on any other record types do not need this treatment.

If you expand the Source Note in the Sources For pane, then every hidden Citation will exist in full, and could be reinstated if you forget to use Edit > Undo Plugin Updates. An evolution of the Plugin could even do that automatically, if this technique catches on, and thereby offer a recovery mode.

[Peter Collier]

I'll a play around with that and see how we go.

Is the plugin case sensitive - do I need to take care to match the capitalisation in the note precisely?

[Peter Collier]

Seems to work well, Mike. Even reporting just the four births and one marriage of a small family it has managed to tidy things up nicely, almost halving the number of references. By the time I run it on my main file it will probably eliminate three or four pages of superfluous footnotes.

Before.JPG (73.31 KiB) Viewed 10632 times

After.jpg (60.31 KiB) Viewed 10632 times

I also think it it is a safer/neater solution this way, as it only hides the citations within the project, rather than deleting them outright.

The footnote regarding the excluded citations always appears in position 2. Thinking about how the plugin works, I can see why that would be the case. Can you think of any way that would leave in position 1 instead?

[tatewise]

I think you will discover that the excluded Source Note entries will appear in various positions. The order is determined by the order of the Citations on the Facts. You are experimenting with a Birth Event, but if you experiment with a Death Event as well, then you will get two One or more Source Citations excluded from Report. included; one for the Birth and one for the Death. It is not possible for a Source Note entry to preempt a Source Citation entry.

BTW: The hiding rather deleting scheme was something I had up my sleeve all along, and could equally apply to the Assessment triggered Plugin. It is always interesting how Plugins evolve once you start experimenting with them.

On closer inspection a couple of things mystify me.
Firstly, how come your entry is all lower case One or more source citations excluded from report.
Secondly, how come so many Citations have become hidden with only one Source Note. Are they all perhaps at the Record level rather than on each Fact.

[Peter Collier]

Ah - the output you see there isn't pure FH as I had further saved the report to PDF, which I then edited in Acrobat in the way I usually do to give myself a better idea of my final result, which is when a little of your capitalisation was changed out of personal preference. (I also dropped the footnote font size by two points and correctly italicised the source titles. Calico Pie really need to give us basic report editing functionality within FH).

I'd discovered the source note had to follow the source citation, which is why I surmised its position among the footnotes couldn't be changed. I thought I'd check anyway as you have vast knowledge of FH's nuts and bolts.

As to why so many citations devolve to one source note, I don't know. The citations are added as below. A lot of the superfluous citations in this family however are for things like the kids' birth certificates supporting the parents names and places of birth so perhaps that's why?

Capture.JPG (201.47 KiB) Viewed 10618 times

[tatewise]

I have realised the reason why there is only one Sources entry for One or more Source Citations excluded from Report. is that FH automatically combines any identical Citations into just one. I think that makes the Sources list neater and more compact than I had expected.

To get that One or more Source Citations excluded from Report. entry to appear first then there must be such a Source Note on <whole record> or Name without any Source Citations on them.

[Peter Collier]

How would I go about achieving that?

[tatewise]

On the Sources For pane click the drop-list at the top to select either <whole record> or Name.
Add a Source Citation as normal and put Exclude From Reports in its Note.