* Format of Sort Dates in GEDCOM to GedSite

[Linda Reinfeld]

Hi Mike -

I didn't think at first that your plugin Export Gedcom File would be helpful to me, but I'm trying it again!
I sent a Gedcom file to GedSite, but there is a problem with the sort dates, and GedSite calls them irregular dates.
I wrote to John Cardinal (creator of GedSite) and he said date format of MAY 28, 1890 is not a valid Gedcom date.
I think it should be 28 May 1890.

I am using the PlugIn Export Gedcom File 3.3. Perhaps there is a date-format box I can check.

Thanks -

Linda Reinfeld

[tatewise]

Hi Linda,
I don't understand how that is happening because the Plugin should only produce standard valid Gedcom Dates.
Are you and John saying EVERY sort Date is irregularly formatted like that, or is it just one?

I wonder if your original Project Gedcom still contains _SDATE tags imported from TMG with that irregular Date format?
If you run the UDF List Plugin as explained in how_to:handling_unrecognised_data_fields|> Handling Uncategorised Data Fields (UDF) does it report lots of _SDATE tags?

If that is not the case, then can you investigate the exported Gedcom file?
One way is to use Notepad to open the ...\Public\Export\Test Export Gedcom File GST UTF8.ged file and search for 2 _SDATE lines, and they should all use the standard 28 May 1890 format.
Alternatively, at the end of running the Plugin use the Windows Notepad button and search as above.

[Linda Reinfeld]

Hi Mike

I ran the UDF in the Tools/Uncategorized data/find uncategorized data. No UDFs were found.
I did a direct import from TMG.
I looked at the Gedcom with notepad.
All _SDATEs are irregular.
Also I found 2 tags that had the same date: Jun 2002.
And per your design (if I understand it correctly), the 1st one had _SDATE JUN 1, 2002, the 2nd had _SDATE JUN 2, 2002.
I also see tags that I had as 2003 have a sort date of JAN 1, 2003

Perhaps there is a date option in your plugin that I have forgotten to check.

Linda Reinfeld

[tatewise]

Hi Linda,
I have found the cause of the Sort Date format problem :-
Tools > Preferences > General > Preferred Standard Date Format: mmm dd, yyyy

I had not realised the Plugin would automatically use that format option when composing the Sort Date.
Sorry

It is fixed in Attachment Export Gedcom File Plugin ZIP file Version 3.3.1 Date 10 Aug 2017.
[Attachment now deleted as fixes are in Plugin Store.]

The Plugin always creates a full Sort Date derived from the Fact Date.
If the Fact Date has no day number, then the Plugin substitutes 1 e.g. JUN 2002 becomes 1 JUN 2002
If the Fact Date has no month, then the Plugin substitutes 1 JAN e.g. 2003 becomes 1 JAN 2003
If successive Fact Dates are the same, then the Plugin adds one day to each Sort Date such that the order of the Facts cannot be misinterpreted.
e.g. If it used 1 JUN 2002 for several successive Facts they might not be shown in the desired order in GedSite.

[Linda Reinfeld]

Hi Mike -

Yes, that worked! Very nice, thank you very much. The sort dates are in the correct format. Wonderful!

But of course, everything is not perfect...
I know my data is not standard FH data, plus I don't know how to do things correctly in FH.
I have a tag (in TMG) called NoteSS. Sometimes it is for 1 person, sometimes it is for 2 people. The 2 people are married, so it does constitute a family event. I have 2 custom facts in FH, one Individual, one Family, both with the same name (which is probably wrong, but I don't yet know what to do about it or how to fix it). They are both Life events. Anyway, this tag in TMG has only a sort date. It comes across fine in a narrative report in FH. But it is undated in the Gedcom, therefore out of order in GedSite.

I'm not sure why it didn't receive a sort date from your plugin. I have tried to investigate on my end but haven't come up with anything - will do more testing. Perhaps FH doesn't like facts that are different (Individual & family) with the same name and I have to change my data somehow.
I am also looking to see if this has happened other places.

Thanks so much for the prompt action re the sort dates. I'm learning how to navigate around - it took a while for me to see your response (I can't seem to get an email notification) and then I have problems with unzipping & installing plug-ins. Just getting used to doing things another way!

Thanks -

Linda Reinfeld

[tatewise]

Linda, nothing ever is perfect, but we just have to accept that sometimes - here endeth the philosophy

There is nothing wrong with an Individual fact and a Family fact both having the same Name.
The standard Census Event has the same Gedcom Tag (CENS) for both Individual and Family versions.
If you need to distinguish them, then you can Label them differently in Tools > Fact Types > Edit.

Even undated facts such as NoteSS should be given a Sort Date (_SDATE) by the Plugin.
But the Sort Date chosen will depend on where that undated fact appears amongst all the other facts.
Don't trust the Facts tab, because that honours the Date and Time Frame regardless of actual Fact order.
Check the All tab and that will show the actual Fact order used by the Plugin to create each Sort Date.
See plugins:help:export_gedcom_file:sort_date_details|> Export Gedcom File ~ Sort Date Details.

You may have to use the up/down arrows on the All tab to move facts into the desired order.
You may also have to use the labelled Note feature to get each NoteSS to appear exactly where you want.

Are you absolutely sure the NoteSS facts have no Sort Date (_SDATE)?
What is your evidence?

You will get Email notifications of Forum Replies if you tick the Notify me when a reply is posted in the Options below the Post a new topic/reply text box.

Sorry about having to unzip my Plugin before installing, but it is larger than the maximum file size allowed for Attachments.

[Linda Reinfeld]

Hi Mike -

Fact NoteSS. Individual Fact. No date (from TMG) HAS sort date

Fact NoteSS. Family Fact. No date (from TMG) does NOT have sort date.

I can tell this because in the GedSite program John Cardinal wrote a browser to look at the Gedcom file. You enter an individual person's ID and it will bring up all entries - Individual and Family.
I can look at an Individual's tags browsing the Gedcom using Notepad, but I don't know how to look for family facts.
In FH the all tab doesn't show family facts. The facts tab shows individual facts and family facts.

I also changed the Normal Time box in editing the fact type. For the Family Fact I said post-marriage. I thought that might help put the fact after marriage. That isn't working even in a report - Individual Narrative - in FH.


Linda

[tatewise]

I have investigated the Plugin and it always seems to allocate a Sort Date (_SDATE) even to undated Family facts.

In FH to see Family facts you need to open the Property Box for the Family record.
The easiest way is in the Focus Window to click on the blue bar showing Marriage details below the Individual spouses.
Another way is in the Records Window on the Families tab, much the same as opening Individual, Source or Media records.
Then in the All tab you can move Family facts up/down relative to one another.

When browsing the Gedcom in Notepad the first record is the HEAD record, followed by the INDIvidual records.
They are followed by FAMily records, NOTE records, etc.
To find FAMily records search for 0 @F as each record starts 0 @F123@ FAM where 123 is the Record ID.
Then you can search for 2 _SDATE tags to find each Sort Date.

BTW:
The Individual Facts tab shows lots of types of Facts :-

• Blue bullets indicate Individual Facts for current person
• Red bullets indicate Family Facts for current couple
• Silver bullets indicate Individual Facts for another family member (Timeline Facts)
• Blue arrows indicate Fact Witnesses

You are correct, setting Normal Time Frame to Post-Marriage appears to have no effect on Family Facts.
I believe that is a bug and should be reported to Calico Pie.
Likewise, a Normal Time Frame of Life or Marriage has no effect on Family Facts.
But any other Normal Time Frame will force the Family Fact to be first or last in the Family record.

[Linda Reinfeld]

Hi Mike -

Thanks for the info about how to see Family info in Gedcom and how to see Family info in FH.

I see that in the Family section of the Gedcom the Marr tags have sort dates.
None of the tags marked 'EVEN' (event) have sort dates, whether they have a real date or not.
Of course if they have a real date (like a census) then they are fine.

I have to analyze my data now, because this routine will be great for the info that comes after death (burial, cause death, obit, etc) - most of mine is only sort-dated. However, with a lifetime timeframe, I see occupations that occur 1 day after someone was born. I did see your reply to someone about making a new occupation tag - maybe I will have to do that. This happens with other facts also. I shall analyze.

I can send you the Gedcom, or post part of it here if you would like to see the no-sort-dates.

Linda

[Linda Reinfeld]

Hi Mike -

Analyzed a little bit more.
In looking at an individual the information in Gedsite corresponds in order with the information on the Facts tab of the individual. For this particular person I have 5 undated in FH (sort date only in TMG) facts that are listed after the death date. These tags in the Gedcom all have a sort date later than the death date even though they are described as Life Events. So that's a problem. But most of those events shouldn't be listed right after birth. So that's another problem.

Ah, it's complicated. The routine works really well for certain facts, and would save me a ton of time. But it doesn't work for other facts, and mis-places the data. I (and other TMG users) really do need a sort-date capability in the direct transfer to FH.

I shall have to count my numbers to see what to do....

Thanks so much for all the help. It is really appreciated.

Linda

[ColeValleyGirl]

Linda,

Do you know where in the sequence of events you want to see those undated facts? if so, move them to that position using the FH facilities and they will acquire sort dates on export that will keep them in that position in GedSite. If (as happens for some facts) you can't move them to the correct place, a manual sort date should do the trick. (labelled Note using Sort Date: label)

Helen

[tatewise]

Firstly I must apologise for the missing Sort Date on Family EVENts
That was a bug in the Plugin fixed in the Attachment Export Gedcom File Plugin ZIP file Version 3.3.2 Date 11 Aug 2017.
[ Attachment deleted as published in Plugin Store as V3.4 in Oct 2017. ]

Regarding undated Facts with say Normal Time Frame of Life occurring after Death is probably because you have not used any of the automatic sorting commands after importing from TMG.
I suspect the import to FH from TMG will retain the order of the Facts as they were in the TMG records.
So if those undated Facts were after the Death Event then that is where they will stay.

Another wrinkle is if those imported undated Facts were custom Facts with no FH Fact Types definition.
They would have no Normal Time Frame reference at all until you created a FH Fact Types definition.
So I am not sure how they would be sorted before that definition existed.

However, try these experiments for cases where an undated Life Fact is after Death Event when viewed on the All tab.

• Use the Property Box cog Menu and choose Sort Family & Events in Date Order that applies to just that record.
• Use the Tools > Re-order Out-of-Sequence Data and select options, that apply to all Individual and Family records.

That should move undated Life Facts that were after Death to just before Death.
It will ensure all Facts are in Date order or at least in correct Normal Time Frame.
The exceptions are undated Family Facts as discussed earlier.

Any further adjustment of undated or identically dated facts can be performed on the All tab using up/down arrows.
BUT if those auto-sort commands above are used again, then such adjustments will not necessarily be retained.
That is where the labelled Note using Sort Date: comes into play. It won't change the order in FH but will in GedSite.

Another thing to consider for such as Occupation is to give them a Date Period stating when that Occupation applied.

[Linda Reinfeld]

Hi Mike -

Thank you, I did reinstall the plugin, and it added dates to the family facts.

Regarding the facts after death date:
This is the order:

Death fact - with a date
Obit fact - with a date
NoteSS fact - no date (sort date in TMG)
Genealogy fact - no date (sort date in TMG)
Burial fact - no date

I can see them as incorrect in the facts tab.
I ran the Tools/re-order out-of-sequence data. It said nothing was out of sequence.
I ran the gear icon for just this person. No change to the data.
The NoteSS fact and the Genealogy fact have been defined as Life events.

You know, I re-converted a (test) database of about 2,000 people, and just stepped through the facts tab of about 200 of them.
No facts appeared after the death date that should not have.....
So there is something about this person's data that is not correct.

I also looked at other tags that I have from TMG
Undated tags:
NoteSS - 1621
Residence 1131 (I am working my way through these, adding a date)
Occupation - 2321 - that's the killer.
However, 3767 Note-Parents tag (identified in FH as just after birth) appear just fine!

I will still work more with the data. But at least some of the non-dated data is working.

BTW I didn't understand the 'Date Period' comment - is that a FH term? I looked it up & couldn't find it....

Thanks for all the help - I have learned a lot.

Linda

[tatewise]

First let's deal with Dates, and remember these are all standard GEDCOM Date Formats.
Open the Date Entry Assistant by clicking the [...] button to the right of every Date box.
The five tabs offer all the available formats (including Period).
Click on each tab in turn and then the Help button for advice about each one.

BTW: Did you ever work through all the guides & tutorials in how_to:key_features_for_newcomers|> Key Features for Newcomers?

Regarding those Life facts that won't sort correctly, I have tried to repeat your scenario, but they always sort as expected.
The undated facts with Normal Time Frame of Life always sort before Death Event even if it has no Date.
If they are moved on the All tab after the Death and Obituary, then the Sort moves them before Death.
To get your effect the must have a Normal Time Frame of Post-Death.

Check the All tab. Are they correctly sorted there?
If so, then on the Facts tab click the Date column header and they should be sorted correctly, otherwise read on...

I presume that NoteSS and Genealogy are Individual facts.
Please double-check the Tools > Fact Types definitions of Normal Time Frame for those two facts.
In the Fact Types dialogue tick Show Hidden on right and review the Label and Name columns for all facts synonymous with NoteSS and Genealogy. There may possibly be some double entries. Perhaps there is both an Event and an Attribute with the same Label but different Normal Time Frame settings.

[E Wilcock]

I had postponed generating a full GEDSITE site from my Projects imported from TMG. I have not needed one and need to take time to learn the programme.

This thread has alarmed me a bit as I paid little attention to fh import of my undated fields. I meant this exercise to try out Mike's plug in and Gedsite - but it has ended with my realising how much I have lost on import.
My data entry of what my father told me was always slapdash and was inserted in undated Tags at the appropriate place in a person's life story.
In fh this chronological sort order has been lost - In fh my great grandfathers fh Fact Tab has 5 undated Events/Attributes following his birth. The first of these is the undated occupation Tag used in the outline version of my sites for use on the web. But the other undated tags are no longer in their original position, including his religion which was dated for his burial - the source of that information.

Although they have been taken to the top, all undated Tags show in fh in the order in which they were sorted (by sort date) in TMG. And my undated Research Notes tag which almost always comes at the end of a record in TMG, still comes at the end.

I downloaded the zip of the latest GEDCOM Export Plug in (Thank you John). I selected Gedsite for the export but didnt alter anything else. The exported file that showed up on my screen showed sort dates had been added but I didnt open or view.

I generated a site in Gedsite 1.8.8. (again out of the box so with no title page or customisation) and the Tags for my great grandfather appear in the same order as on the fh Facts Tab. Records created or edited in fh are fine.

So the plug in works fine. My loss of data is from the original import.

I went back to TMG and saw that I have actually lost a lot by the loss of sort dates. Not in the sense of losing the data, but in the meaningful narrative.
It looks so bad that I may eventually decide to export records updated in fh back to TMG.

[tatewise]

Evelyn, it sounds like you have not taken on board the tactics discussed earlier in this thread.

In particular the use of Tools > Fact Types > Edit > Normal Time Frame to control the position of undated Facts.
For example :-
To ensure undated Research Notes fact tags come at the end, set the Normal Time Frame to Post-Death.
To ensure undated Religion fact tags come near the Burial, set the Normal Time Frame to Death.
Likewise for other custom Facts, but standard Facts cannot usually alter their Normal Time Frame unless you use an advanced Tools > Fact Types technique.

After that, if undated Fact tags are still misplaced, then either give them a specific Date, or use the up/down arrows to move them into position, but beware that using the auto-sort options may disrupt such manual positioning.

It is unfortunate that the _SDATE Sort Date values are not imported from TMG into FH, because then a Plugin could sort the Facts into _SDATE order.

[E Wilcock]

Thanks Mike. It might be quicker to re-order manually than to return the Project to TMG.
I did read all the advice - Just hadnt grasped till I tested your new plug in how much it applied to me.

I cant programme the Tags e.g. Religion to go with burial as it will be different for each person, depending on the source. Some one else ex TMG has mentioned that they dont want to give dates unless the date is from a source. My data entry is due to negligence rather than being as careful as she!

I wouldnt do any automatic sorting either.

As you know I have several thousand names now in new projects started in fh - no problem with them. And love the data entry.

I have 4 trees imported from TMG and will have to do a search for undated Tags to see the extend of my problem.

But Calico Pie really need to address this - Do I need to write to them? I am an every day family historian rather than the expert organiser of large place or name projects. And it isnt good that my work from years ago is so affected.

[tatewise]

Yes, do report your problem with sort dates for undated Facts to Calico Pie.
See how_to:about#problem_reporting|> Problem Reporting.
Remind them that TMG, RootsMagic, and GedSite all support the same _SDATE tag.

[Linda Reinfeld]

Hi Mike -

I have spent the past few days going over my data to see the extent of the problem with no sort dates.

The problem is larger than I thought. I have gone through at least 800 people. I added more people to my test project. 250 were living, so we're down to 550 people. Over 20 people have multiple facts sorted after death dates that have a 'Life' time frame. All of the facts that occur after death date also, in other people, occur before death date. At least 10 different fact types, including employment, religion, residence, graduation, notess. These are all undated tags in TMG with a time frame of 'Life' in FH. I cannot find any pattern to those that work & those that don't work.

I have never manually sorted a fact in FH. I am still working on importing from TMG.

Although I haven't checked extensively, the facts in the All tab seem to be sorted the same as in the Facts tab. I understand that the family facts will not sort correctly even though they have a Life time-frame. I have discounted those instances in the reporting of the problems.
I have always looked at the hidden facts along with the non-hidden facts in the tools/fact types section. I have checked the time frame for all these facts. My guess is that because FH doesn't import sort dates the facts are quite mixed up when they arrive in FH.

My data consists of over 39,000 people. As you might guess this is very discouraging. I'm not sure there is any answer to this, due to the non-import of sort dates. But, thanks for all your help. I am certainly learning a lot about FH!

Linda Reinfeld

[E Wilcock]

I too have discovered it is a worse problem than I thought.

I reported to Calico Pie who replied that "importing of the SDate field is something we already have logged for consideration".
Given the size of my problem I felt this was somewhat casual - and I wrote back rather crossly to point out that any of us with undated fields were not being offered adequate import from TMG which we had been led to expect.

In my innocence I didnt realise that sort dates which I connected with ordering the children within a family, included any undated Tags.

I have plenty of undated events. My tree was copied from old trees draw out on paper - They gave some birth and death dates but rarely dated a marriage. I also have undated notes and comments. I created undated Occupation and Residence in order to use them on my outline BMD web site.

This means that when I run a fh query to locate undated Events it seems to include every individual.

I have imported three of my family's trees from TMG to fh. 362, 576 and 1622 people. They were the ones I wanted to add to or work on. The fh date last edited column shows so much updating that I dont think it feasible to go back and re enter that data into TMG.

I would welcome any suggestions on how to find and correct out of order data. It could well be that there is little to correct, as I havent noticed anything in what might be termed average entries providing only B M D -

But have I really got to work down an alphabetical list of people opening up and sorting the facts tab for every record?

[tatewise]

Linda, can you post a few examples of GEDCOM snippets showing Life facts occuring after Death Event, together with a few showing some of the same Life facts occuring before Death Event.

If there is no other explanation, then it would appear to be a bug in FH, bearing in mind we have already discovered that Family facts do not honour some Time Frame settings.

Evelyn, given that FH does not support Sort Dates, it is impossible for any automatic mechanism in FH to determine if undated Facts are out of order ~ how would it know?
Without a Date the only other sorting mechanism relies on the Normal Time Frame set for each Fact Type. So currently, the only way to position undatd Facts is manually.

[Linda Reinfeld]

Hi Mike

I looked at the Genealogy, NoteSS, Religion, Residence facts.
I am attaching snips of the Properties Box/facts tab.
All the facts are identified as Life.
I have attached a snip of the 'bad' order and then the 'good' order
Let me know if this is OK.

Linda Reinfeld

[Linda Reinfeld]

Sorry, I can't seem to attach, just post 'inline' and can only do 3.
Here are others:

[Linda Reinfeld]

The last ones.
Notice the Residence_Good_family - that came out as a Life event before death, even though we said there was a bug with family Life events
I have identified in the file name the fact and whether it is 'good'

Linda Reinfeld

[E Wilcock]

Linda - I dont want to hijack your thread.

First I will reply to Mike- that no I did not expect any automatics re-ordering. Just wondered the best way.
I started by looking at the Project holding my husbands tree 575 individuals. Going though the ca 200 records that have been edited in fh, I couldnt find any errors - The reason I remained unaware of the problem for so long may be that my particular use of undated fields (usually occupation and residence for my outline website trees) imports well to fh. I dont at all mind those two undated facts coming first and the undated research note coming last i.e. after death.

It looked to me as if re-ordering might be needed only where a person record included several undated Events, including two of the same type.

I devised a custom query (thank you for teaching me Mike) to find all undated events and then sorted it on the column giving the names of the individuals to whom those events belong. That list shows me anyone who has multiple events. Or where an unusual Event is undated. I went through checking and had to make only one adjustment.

That is one Project cleaned and I am pretty confidence the two others will be sorted the same way, by running that Query.

Looking at Linda's images, I thought I might look at another TMG Project opened for reference in fh but not really edited. It has only 635 and is a Project I used for previous academic research - meaning I used it like a notebook to collect and paste in a lot of miscellaneous information about university staff and to show relationships by marriage.

I ran the custom query on this a did find out of order data. I will snip this in a second post. But when I looked at the same record back in TMG, I saw that the order there was already very odd. I dont want to post a snip from TMG as there are living people there. But I have undated fields in this order
Daughter
Daughter
Marriage
Residence
Education
Occupation
Title
Academic Qualification
followed by the only three dated Events
Birth
Will and
Death.

Yes - I know one should not throw in data in this fashion - but it is the way I work.