* Search and Replace Plugin, filter on fact

[jimlad68]

tatewise - just a thought on your excellent Search and Replace. I have been comparing it with the Find and Replace function of TMG Utility
http://www.johncardinal.com/tmgutil/findandreplace.htm
I think your Search and Replace has features that the TMG utility does not, but one feature I cannot find in your Search and Replace is the ability to filter the fact/events type.

What am I trying to do? (I can do this in TMG before I move my data, so I can manage without it in FH, but might be a usful feature for the future).

I want to give the same date to all records of a custom fact. In this case something like 0150 so I can use the dates to ensure they appear at the beginning of reports and if I need to can easily tidy up with a word macro. In TMG this is not a problem as it uses Sort Dates, but does not export in GEDCOM.

In TMG utility I seach on regular expression ^(.*) to find anything including blank, I can then filter down to a large selection of fields (Event date, event memo, name date, source etc etc), then further filter to event/fact/tab type (birth, death census custom tab etc etc).

At present this seems to work with your Search and Replace (except see P.S. below) by just selecting date fields + fact notes, but I need to skip thousands of records so it would be easier just to edit/macro the GEDCOM file or even write a plugin.


P.S. With Lua expressions (or come to that MS Regular Expressions) I can't find a way of finding nothing on its own (empty, not null), is there an expression for that or does the program receiving it have to be written to accept nothing? E.G. If I wanted to change only empty fields in your Search and Replace. With your Search and Replace if I use plain text mode with nothing in the search field, it finds nothing, yet I know there are items with no DATE in the GEDCOM, or perhaps your SR does not SR/Create them as they are not in the GEDCOM to start with., i.e. you would have to create rather than replace.

[Jane]

If you simply want them first why not just make them pre-birth facts and not put a date on them? I think that should keep them at the front of reports and the fact list on the property tab.

[tatewise]

Your suggestion may have some merits, but I need convincing there is no other solution.

BTW: I suspect when you say "selecting date fields + fact notes" you mean "selecting Date Fields + Fact Dates".

As Jane says, in FH V5 you can use Tools > Work with Fact Sets, select the Custom Fact, click Properties, and set Normal Time Frame to Pre-Birth or Post-Death to make that Custom Fact come first or last in the Date sorted list.

A subsidiary question is why do you need those Custom Fact to be first in a Report?
There may be another solution in FH that TMG does not offer.

Regarding your P.S.:
Search and Replace should find empty fields if the Search box is empty, but I have just found a minor bug that fails when Whole Words is selected (will be fixed in next version).
However, Part Words works OK, and Search for ^$ in LUA Pattern Mode works OK.

Nevertheless, as you suspect, empty fields rarely exist, because FH deletes them on Save.
Also the Plugin performs Search and Replace and does not create missing fields.
In the GEDCOM hierarchical data structure, there are many potential sub-fields in every Fact and Record.
Those blank/missing sub-fields simply do not exist, the whole sub-tree is pruned away.
This is different from a conventional database Record structure where all fields exist as soon as the Record is created.
BUT, those blank/missing fields can be discovered using Queries.

[jimlad68]

Jane, That works great while inside FH, but it does not seem to export the prebirth info to the GEDCOM, other than placing it at the front of the items. Fortunately Rootsweb trees just seems to list the items in the order of the GEDCOM, regardless of the date of the item (I generated an item without a date in the middle of dated items).

So, that would seem to be a very quick solution for my existing scenario, and would not leave strange numbers or "date queries" in reports, only issue is I may have more than 1 "pre birth" item, but they would at least all be at the beginning or the end for post death. For certain other scenarios I like your idea in another post of using an interpreted value.

I suppose it also comes back to how do you get sort dates with a standard GEDCOM, but I think that has been discussed before.

thanks again

[jimlad68]

Mike, re "selecting date fields + fact notes", yes I got a bit mixed up, but I suspect I should be suggesting "the ability to filter the fact/events type for Short+long and Distinctive i.e. as many fields as possible. Whilst you might not think it useful for my suggestion here, I think it would be far more useful to refine the filter for text fields. However, I suspect this could be difficult to expand, so I have no expectations. One thing I have learnt in IT, some things sound easy and end up being difficult, but also vice versa, so its always worth making the suggestion.

You ask "A subsidiary question is why do you need those Custom Fact to be first in a Report?" I have some custom facts. One is for a very brief description or distinctive feature or "link" to help as a quick reminder at the top of a report or especially in a chart. Another is a fact for "thinking aloud research or queries or doubts" that would effectively be asking any viewers for further info. Also my own more private checklist for that individual.

I have certainly learnt more about GEDCOM structure since moving to FH.

cheers

[tatewise]

Yes, I assumed that was what you meant.
The current Filters all choose classes of field, but many of them can occur in any of the 9 Record types (Individual, Family, Note, Source, Repository, Submitter, Submission, Multimedia, Header).
My interpretation of your suggestion would narrow the search using scope Filters to first choose a Record type, and for Individual and Family types allow any Event/Attribute to be chosen.
The class Filters Search and Replace would then be confined by the scope Filters.

Regarding the order of Facts (Events/Attributes), in Charts/Diagrams the Text Scheme determines their order in boxes, regardless of their GEDCOM Record order.

If you have more than 1 Pre-Birth Fact, then their order is determined by their GEDCOM Record order, which can be adjusted in the Records Window, and in a Property Box on the All tab or on the Facts tab with List in Record Order button set.

As you said earlier, the Pre-Birth setting is not exported in the GEDCOM, which is because there is no standard field for it, and if a custom field were used then other genealogy software would probably ignore it.

[tatewise]

On second thoughts, the Pre-Birth and Post-Death setting is effectively exported in the GEDCOM because they are reflected in the order of the Event/Attribute tags in the Record.

Referring back to your original posting, I have implemented a revised version 2.3 of the Plugin available from my OneDrive at Search and Replace.
If there are any problems then the Plugin Store version 2.2 can be reloaded.

It adds a Search Scope global filter that restricts the Plugin to any one Record type or any one Event/Attribute type.
Beware that this can lead to some impossible searches, such as scope Source Records with only Fact Place fields ticked in Basic/Extra Filters, because Fact Place fields can never be found in Source Records.

Feedback comments will be welcome, but note that the Help & Advice has not been updated.

[jimlad68]

Mike, that works a treat narrowing down the Search Scope per fact type.

Although still very useful, I suppose if I start to use method1 sources I will have less need for it. So my next suggestion is, and I don't expect this to happen:

When the Search Scope is Source Records (SOUR), that could be further filtered when the source affects a single fact type.

[jimlad68]

Jane, Mike,

A further thought on ordering Facts.

I have noticed if I set a date at say 0150 and use "pre birth" etc for the Time Frame of a fact, Reports do not display the strange age note e.g. "(age -1711--1712 (!))" when calculated ages are requested in Report Options, a very nice feature.

I'm still undecided, but because of this in all or just certain situations I could use both Pre-Birth etc and a "dummy sort year" to keep my facts in order, even though as Mike points out that "the Pre-Birth and Post-Death setting is effectively exported in the GEDCOM because they are reflected in the order of the Event/Attribute tags in the Record".

[tatewise]

Jim requested:

When the Search Scope is Source Records (SOUR), that could be further filtered when the source affects a single fact type.

What are you trying to achieve?
May be a Query would give what you want?
Source Records are cited by much more than Facts, and there are many other linked relationships, so to be complete the Plugin would need to offer them all!!!!!!!

[jimlad68]

Yes Mike, I appreciate it is getting a bit messy, so I will rephrase:

"When the Search/Change Scope is Source Records (SOUR), it be further filtered by Fact Type". I accept that there are other filter possibilities.

For the numbers likely to be involved, I am sure I could run a query and amend manually, or run the Search/Replace plugin and skip unwanted items.

Thanks again for the latest amendment, I have not had any problems with my tests so far.

[tatewise]

That still does not define a meaningful algorithm, because if only Source Records (SOUR) are being searched then by definition NO Facts are involved, because they are Individual Record (INDI) or Family Record (FAM) components.

You presumably are wanting to search within the subset of Source Records (SOUR) that are CITED by a nominated Fact type?

[jimlad68]

OK, so a search is done for text in Source notes, that could pick up say 50 Sources, those 50 sources might be linked to 200 facts between them, but say 6 Fact types . The idea is to only find the sources that relate to 1 of those Fact types. I appreciate it might not "define a meaningful algorithm", but I think it is a possible scenario.

Please don't worry too much about it as I'm sure for the likely instances it can be done manually via Query or skipping sources via the standard Search/Replace.

[tatewise]

I do understand exactly what you are asking for, but may I explain a bit more about how GEDCOM structures work.

When it comes to Facts and Source Records the linking is only one way, from Facts to Source Records.
So from a Source Record it is not possible to directly follow any link to a Fact.

Unfortunately, it has to be approached from the opposite direction.
All Facts (or a chosen subset) have to be interrogated to see if any are linked to a particular Source Record.

The same is true of links from Source Records to Repository Records.
Similarly from various Records and Facts to Note Records.

Interestingly, the links between Individual Records and Family Records to form Parent-Child and Spouse relationships are bidirectional.

So for example when creating a Query to find what you are asking for, you must start with a Fact Query to filter on Fact type, and then filter the links to Source Records based on the text in say Note fields.

[jimlad68]

That makes sense, hence Jane's List All Citations for a Source and your amended version List All Citations for Any Sources take a relatively, long time to run.

[tatewise]

Yes, my Plugin exhaustively searches the entire GEDCOM database to find all Citations.
i.e. It searches every data item at every level in every Record type.
This is necessary because Source Records can be cited from so many different places.
Jane's is not so exhaustive, and only checks 'INDI', 'FAM', 'OBJE' & 'NOTE' Record types, which finds most Citations.

Also you can combine two Plugins to achieve interesting results:
e.g.
Run the Search and Replace Plugin with Source Record (SOUR) global Scope and any Basic/Extra Filters.
Use the Search ONLY button and Report all or any manually chosen Records.
This creates a Result Set of the chosen Source Records.
Manually select whichever Source Records are of interest.
Run the List All Citations for Any Sources Plugin.
This creates a Result Set of all Citations linked to those selected Source Records.
Click on the Data Ref or Citing Item column header to sort all similar Fact Types together.
Double-click on any one item to open the Property Box with that item selected.

This may resolve the search you were originally trying to achieve.

If you become proficient at writing Plugins, it is quite easy to create a custom Plugin for a specific Search and Replace.
This is partly because it is focused on specific Records & links, and partly because only a minimal (if any) user interface is needed.

[jimlad68]

yes works a treat, by sorting on the Data Ref column in the results, it is easy to choose just 1 Fact type etc. It's a matter of remembering all the flexible ways FH queries/plugins can be used.

One thing I have not fathomed out on the query results is how to select none-adjacent multiple items. Whilst multiple adjacent "cells" are easy to select by various methods (e.g. Shift+Home), the Help would suggest using say CTRL+mouse click is only possible on the Records and Diagram Windows.

[tatewise]

Yes, it is a bit tricky, because Result Sets do not support the usual Ctrl-click select mode.

There is a workaround for the Records in a Result Set (but not I think for other items):
Select all the adjacent Records that include the ones you want to choose selectively.
Use the Query Menu > Add Selected Cell Records to Name List.
Choose an existing Named List or create a new Named List.
In the Records Window open the Named Lists Pane using the toolbar button (next to the green triangle).
Select the Named List used above, and now you can use Ctrl-click to individually select Records.