* Case-sensitive file checking

[Mark1834]

I'm looking for a quick method to check that a file exists, but it must be case-sensitive in Windows. Both the traditional io.open(Filename) from lfs and fileExists(Filename) from fhFileUtils are case-insensitive, following normal Windows convention. I can do it by comparing the case-sensitive filename with the list of files in the parent folder from lfs.dir, but it's a bit long-winded. Is there a simpler way?

Background - a custom audit plugin that compares Media Records, linked files and actual filenames on disk and corrects any mismatches using the name of the file as the master copy. It is only run occasionally, so it's not worth dozens of lines of code to shave seconds off the run time, but if I can do it in one or two lines it is worth the change.

[ColeValleyGirl]

I haven't tried it, but one of the other methods of the Windows Scripting filesystem object might do the trick. You can steal some code from fhFileUtils and try the GetBaseName method.

[Mark1834]

Thanks Helen. I managed to test that without any problems, but none of the methods in FileSystemObject appeared to be case-sensitive (perhaps not surprisingly for a Microsoft tool ).

I restructured the plugin slightly and saved the results of each lfs.dir in a table for reuse rather than rerunning for each media file, and that fixed the problem of longish run times (>10 s to <1 s). Fortunately, I don't have any files with extended characters in their names, as I suspect that would make manipulation of their names in tables more complex.

The plugin does four things:

• Silently correct any case-mismatch between the Media Record file and the actual disk file. The normal FH Tools>External File Links misses these. I queried this with CP, but they only intend the tool to identify non-existent files so for them it is working as expected.
• Identify any potential typos in file names such as multiple spaces or malformed punctuation and prompt for new name. Relink the new file and update record title automatically.
• Identify any mismatches between the linked file name and Media Record Title and correct after a confirmation prompt. If a media file is renamed, and Tools>External File Links used to recreate the link, it does not update the Title field.
• Silently convert any "jpeg" reported formats to "jpg". FH is annoyingly inconsistent in which it uses, and it has been so since at least V6. It's been reported to CP (about 2 years ago) but never been corrected.

This is the other way around to Joop van Beek's recent store plugin. He manipulates the file to match the record, and I start with the premise that the file is the master and manipulate the record to match.

[ColeValleyGirl]

Sorry my suggestion didn't help. The extended characters might not be a issue if you use the utf8 library.

[tatewise]

The multimedia Format field confusion is compounded by the GEDCOM specifications:
GEDCOM 5.5 Page 47 MULTIMEDIA_FORMAT:= [ bmp | gif | jpeg | ole | pcx | tiff | wav ] i.e. FH v6.2
GEDCOM 5.5.1 Page 54 MULTIMEDIA_FORMAT:= [ bmp | gif | jpg | ole | pcx | tif | wav ] i.e. FH v7.0
GEDCOM 7.0 Page 68 FORM (Format) The media type of the file referenced by the superstructure. This should be a valid media type as defined by IETF BCP 13. A registry of media types is maintained publicly by the IANA.
https://www.iana.org/assignments/media- ... html#image lists jpeg not jpg and tiff not tif.

Should the FORM field conform with GEDCOM or match the actual media file type?
I agree that FH is inconsistent.

[Mark1834]

Philosophically, I prefer jpeg as it is the officially defined version, but recognise that jpg is far more common (thanks a bunch, Microsoft). To minimise renames I have standardised on jpg for both file names and the FH descriptor, but it would be perfectly possible to always use jpeg as the description of jpg files. Having both in FH makes proper sorting of the Records Window a PITA...

[Jane]

BTW You can get the cased version of the name
e.g

Code: Select all

require "luacom"
fhInitialise(7);

local fso = luacom.CreateObject("Scripting.FileSystemObject")
local sFile  = fso:GetFile('C:\\Program Files (x86)\\Family Historian\\Program\\Lua\\fhfileutils.fh_lua')
print(sFile.Name)

returns

Code: Select all

fhFileUtils.fh_lua

[Mark1834]

Thanks Jane. I’d not used luacom/FSO before and was looking for something that returned a name string directly. I’d not considered the extra step of object.property to return the property value. That’s a worthwhile modification for simplicity, and is probably more compatible with any future extension to extended characters.

[Mark1834]

If I’m using luacom/FSO for a case-sensitive file checker, it probably makes sense to replace fhfu.fileExists() by fso:FileExists() and os.rename() by fso:MoveFile(), so I’m doing things in a consistent manner.

There seems to be a lot of overlap between fhFileUtils and luacom/File System Object (is one basically a wrapper for the other?) but I don’t have a good overview of the pros and cons of the different approaches and which is best suited where (or where the standard os and lfs libraries are perfectly adequate). At the risk of adding to workloads, is it worth a summary page on the KB? Currently it says they exist, but doesn’t compare and contrast.

[ColeValleyGirl]

Yes, one is a wrapper for the other except it also wraps some other objects to handle reading and writing ANSI utf16 and utf8. It's intended to simplify using the relevant objects so that people don't have to work out the syntax of some badly documented Microsoft features, and thus save plugin authors some time and brain cells. Like snippets but even easier to use. It was implemented to handle utf file names which the standard functions in lua and lfs don't, so it will handle a broader range of other languages than ANSI.

Do you think it would be worthwhile to include getting the case sensitive file name as a function or is it not likely to be widely needed?

[Mark1834]

Difficult to comment on the general usefulness. I think it is, but it happens to fit my way of working well, where I give the source file a highly structured name and allow that to dictate properties of the media record. If you do it the other way around, it might be less useful.

From the extremely limited number of examples I’ve used, I don’t find a great difference in the ease of use or documentation quality between FileSystemObject methods and fhFileUtils methods. It is potentially confusing having both though.

At the risk of being slightly controversial (perish the thought ), I wonder if the emphasis should actually be the other way around. I suspect quite a few potential plugin authors have probably dabbled in MS Office VBA, so will be more familiar with that and see the overlap with FSO, rather than hiding it in a new Lua library, which people will be less familiar with. But maybe I’m just influenced by my own previous experience and route in.

[tatewise]

I believe that FSO will work in FH v5, v6 & v7 whereas fhFileUtils is only available in FH v7.
So if Plugin authors want backwards compatibility then FSO is the way to go.
I've used FSO in at least two Plugins that are fully backwards compatible.

[ColeValleyGirl]

Mark/Mike -- for experienced vba programmers/those with existing plugins who need to maintain backwards compatibility, it may well be easier to use filesystem and adodb.stream objects -- once they've been pointed in the right direction; even experienced vba programmers may never have used those objects.

However a lot of users/potential new plugin authors may not have vba experience and almost certainly won't care about backwards compatibility, so the library will be useful to them (as well as -- IMO -- producing simpler code that's more readable.)

Re backward compatibility, I didn't have time before the last FH update to review the code for Lua 5.1 compatibility, but it's on my to-do list -- if only so I can do away with the complex code I've taken from Mike to let me read and write UTF-16 files.

[Mark1834]

It's incidental to the main subject of this thread, but I've changed my mind on media formats after doing a bit more digging. If you create a new Media Record, the list of formats presented in the drop down exactly matches the GEDCOM 5.5 list Mike described above. So FH7 has stuck with the official formats, and not implemented the non-standard ones introduced in GEDCOM 5.5.1.

The long-standing bug is to frequently record jpeg as jpg when importing media, so I'm sticking with the official version, and will convert jpg to jpeg, not the other way around. I made the bulk change in a text editor (2 FORM jpg to 2 FORM jpeg) in order to avoid thousands of records having the same revision date!

A trivial point perhaps, but standards and consistency matter...

[Ron Melby]

require "luacom"

local fso = luacom.CreateObject("Scripting.FileSystemObject")
local sFile = fso:GetFile(<somefile>)

Jane posted this a little earlier, am I to understand this only works on V7 and not on 6? because I get errors from luacom.

[tatewise]

As stated in Lua References and Library Modules the luacom library is OK in FH v6.
I use luacom.CreateObject("Scripting.FileSystemObject") in some of my plugins that run in all FH versions.

Exactly what error messages are you getting?