* Gedcom Export to Legacy FT

[tatewise]

Those codes ÙCbÙD are the «b» codes, but you must be using an editor that does not recognise the character encoding.

Did you export in ANSI or UTF-8 encoding?

In the Windows Notepad editor both encodings display the «b» codes OK, and they import into FH OK.

BTW: Although Legacy appears to support UTF-8 encoding, it actually does so only for the ANSI range of characters, and NOT the full UNICODE set.

[ronk]

Interesting...I was in RJ TextEd 'text' mode, and just tried loading the gedcom in MS Wordpad and Word, and none display the <b> code. Tried Windows Default, Western European.

I must be missing something.

Exported in ANSEL, first in the dropdown, neither ANSI nor UTF-8. Guess I should pay more attention to that. Wiki indicates ANSEL is for Bibliographic use. So which should I have used for my initial export to FH?

Ron

[tatewise]

You must not use ANSEL because it only supports about 200 of the 256 ANSI characters, and « and » are not among them.

We have not had much dialogue with Legacy users until now. But in the light of current investigations I have updated the how_to:import_from_legacy|> Import from Legacy advice. You might like to review that and make comments here for further improvements.

[ronk]

OK, understood. So likely I may have to redo my initial export. Was considering doing that anyway. I can't be sure, but doubt my data uses many of the additonal characters. I will look over your Knowledge Base article carefully before doing a second migration, if I choose to do that.

Is the FH unicode gedcom it's only 'database'? I looked over all the related files and found nothing else. If so, how does it do the processes so quickly, e.g., relationships, and switching file root? It seems instantaneous. Legacy is rather slow in this regard.

Ron

[ronk]

For my usage, don't worry about the HTML. I just ran a test and was able to convert the few characters I used quite easily. Looked fine in Legacy. However other users might use more of them, but I doubt there are many users making the 'round trip' as you put it, like I am.

Ron

[tatewise]

Try ATTACHMENTS Export Gedcom File Version 2.5.4 Date 03 Mar 2016.

It fixes the FILE~REL problem, excludes CHAN timestamps from Notes if you select Update 1 CHAN: Remove entirely, and converts < and > to « and » as appropriate, but may not catch them all.

Yes, apart from customisations in C:\ProgramData\Calico Pie\Family Historian\ and Windows Registry the only 'database' is the GEDCOM file and of course all the Media files.

FH loads that GEDCOM file into memory and builds its own internal relational data structures. That is why it is fast, because it does not rely on some relational database product. Only when you click Save, or at the Autosave intervals, does it write to the GEDCOM file.

[ronk]

Fascinating, confirms what I thought that FH was reading the gedcom file into memory in order to have the relational structures it needs. Cleverly written for speed. I suppose memory may be a constraint with really large files then, but there's always SSD's for paging/cache file.

I have SSD for software, HDD for data. Experimented with putting all media and gedcom on the SSD last night, and saw slightly better read on media related screens, but not worth Mainly in some of the graphical displays pulling up images.

I will test the revised Gedcom Export to Legacy today. The only edit I should have to do is removing the 'caption note:' records.

Thanks,

Ron

[tatewise]

I doubt if large Project Gedcom would be a problem.
I have some test Projects with ~100K Individuals and ~40K Families and its Gedcom file is less than 100MBytes.
With RAM typically many GBytes these days it is unlikely to be a constraint, and all PC use virtual memory anyway that overflows onto disk, which by default is your SSD.

[ronk]

After export with Gedcom Export to Legacy FT 2.5.4. The CHAN records are indeed removed.

But, the <x> I looked at are not converted.

Ron

[ronk]

After I properly edited out the 'caption note:' all worked fine.

Caption labels removed successfully from LFT Export, but.....the < x > single angles were not reverse mapped for some reason. T can live with that, since TNG is OK. For FH wish list, would be nice to be able to bold, underline, etc text though.

I'm now planning to rebuild FH from a fresh Export from Legacy itself, following your revised guidelines.

Thanks!

Ron

[tatewise]

Could you give some specific examples of <x> not converted including their Gedcom tags.

[ronk]

Mike,

I believe all of the html code is found in notes. Have shown here examples for 2 individuals. Nearly all are in the 'General' tab of the Legacy Individual screen. For each individual, have imaged the following:

1) The original Legacy note display before migration
2) The migrated FH note display
3) A snippet from the gedcom from the Gedcom Export to LFT.

I2772.jpg (223.25 KiB) Viewed 9790 times

i278.jpg (252.4 KiB) Viewed 9790 times

Let me know if you need anything else.

Thanks!

Ron

[tatewise]

Thanks Ron, that identified a long term minor bug in the code that was easily fixed.
So try ATTACHMENTS Export Gedcom File Version 2.5.5 Date 04 Mar 2016.

That should convert most HTML <tags> without converting most genuine < or > characters, but it is not 100%.

[ronk]

Almost got it. Gedcom Export LFT 2.5.5 is finding the <tags>, but instead of outputting « and », the re-mapping both sides to ?tab?, or Hex 3F.

1.jpg (160.32 KiB) Viewed 9740 times

2.jpg (180.75 KiB) Viewed 9740 times

Had a another quick question. For 'uncatergorized items', does one have to modify the FH gedcom to remove/correct them, or can they be fixed in FH itself? For example, had some items where SEX was undefined, and came through as _SEX U items. I did put female or male in FH field, but the error field remains.

Ron

[tatewise]

Sorry, got ANSI/UTF-8 encoding confused.
Try ATTACHMENTS Export Gedcom File Version 2.5.6 Date 04 Mar 2016.

Regarding UDF see how_to:handling_unrecognised_data_fields|> Handling Uncategorised Data Fields where it explains how to delete UDF.

FH never deletes UDF of its own accord, because they are Uncategorised Data Fields, and it has no idea what they are.

In your SEX example, only you know what _SEX U means, and that entering a SEX value corrects it, so you must delete the UDF. BTW: If you want the SEX left undefined, then just delete the UDF, because in FH/GEDCOM no entry means undefined.

[ronk]

Html looks good!

Also, thanks for advice on uncategorized data. Have less than 100 items, pretty good considering ~14000 individuals and 4000 families. That means a very good match to original Legacy file, I think (and had no errors importing back to Legacy as secondary either).

Thanks very much! Tested for LFT and TNG.

Suppose you will be able elevate the 'pre-release' status soon then?

Ron

[tatewise]

If nothing untoward comes to light over the weekend, then hopefully can publish to Plugin Store early next week.

Feel free to experiment with options that may be useful to you.

[ronk]

Legacy had options on export to exclude all source records. Does your Gedcom Export plugin? I see some options of source handling, but didn't seem to do what I wanted. Does the Export to Legacy FT (and TNG) have such an option?

Otherwise would need an editing macro.

Thanks,

Ron

[tatewise]

Ron, FH is much the same, click the Plugin's own Help and Advice button to see the plugins:help:export_gedcom_file:export_gedcom_file|> Export Gedcom File > Help and Advice If a subset of data is needed,... advice.

[ronk]

Sorry.
Before posting (and pestering you, since you seem to answer most posts) I did look at the 'Help and Advice' and Knowledge Base articles, but I do confess to tend skim reading. The only references to Source in the Export plugin were in the Extra Options 'Source type 1_Type' which I tried, and the Other Options 'Source Record 0 @S....' box which is unticked and would just move to a Source record. There's also references to Submission and Submitter records which I didn't think were relevant.

I guess the nomenclature is quite different and I apologize if I missed it.

Ron

[tatewise]

The point is that conditional export of Source records is not covered under the topic of Sources, because FH allows most records (and some fields too) to be excluded in its File > Export > GEDCOM File and File > Split Tree Helper commands. So you can produce various subsets of the database. Maybe Legacy has not so much flexibility?

That is what the If a subset of data is needed,... advice on the first page of Help and Advice is all about. I rarely put functionality into Plugins that already exists in FH. I know in this case it means a two step process to export the GEDCOM without Sources and then use Plugin to convert for Legacy/TNG, but that is a fairly rare requirement.

[ronk]

OK, will take a look at the procedure. Makes sense since the FH Export procedure has exclusions. I will export to a 'unicode' gedcom, excluding source, then open in FH and use the plugin.

Actually, Legacy is very versatile in regards to exporting by tags, individuals and descendants, group subsets, etc. and one could export either to 'Legacy' type file or to gedcom. That is the way many years ago I gradually built up my genealogy db from many trees, and then even excised some that were too distant (5th cousins x times removed, etc) and the Legacy tagging allowed me to do that. And it has field exclusion capabilities but perhaps not commands to accomplish that.

Case in point, you might recall I'd entered some 'new data' into FH. Then, since I'd missed the specific guideline for Export from Legacy (ANSI vs. ANSEL), I rebuilt FH gedcom from the original Legacy db file, losing that data. So, I tagged all the FH records I'd added in the Legacy file exported from the 1st FH file, exported them to a gedcom for the 2nd FH gedcom build, and then used the Merge/Compare of FH to bring the data back in. Even the media was brought in perfectly.
Hope that rather lengthy story makes sense, but it worked really exceptionally well. I believe I could also have taken my earlier FH gedcom and Merge/Compared it with the newer FH gedcom as well. I just was more familiar with Legacy 'tagging'.

But I shall check out and try the 2-step process you're referring to.

Thanks!

Ron

[tatewise]

Similarly, FH has extensive flexibility in selecting Individuals to export using Relatives, Queries, and Named Lists. The Split Tree Helper has options to delete Note fields, any selection of Facts, record Flags, etc.

[ronk]

The suggested procedure worked very nicely with the two steps. At first I thought I lost 29 families, but realized the File-Export default ticked to exclude one member families. Unticked, then all matched to earlier statistics in TNG except no sources.

I understand and agree that the plugin need not do what the main FH does already. The gedcoms are processed so speedy that it only takes an extra few moments to do.

Just an observation that the records window doesn't have a tab for families, yet I know they are accessible. Any reason why FH omitted it there?

[tatewise]

I am sure I have advised you work through how_to:key_features_for_newcomers|> Key Features for Newcomers where you will find the answer regarding Families under Records & Fields & Multimedia.