* FHUG Searching

[davidf]

I'm not sure that there is a great mystery - as I expect you know. But for those who don't.

example.com is a domain

www.example.com used to mean "The World Wide Web physical server (think computer with a big disk) connected to example.com domain". Pedantically I suspect on some setups it still does. www.example.com is a "host name" defining where stuff is located.
ftp.example.com would mean the "File Transfer Protocol server sitting on the example.com domain" - a server specifically for file downloading (still found in some places for big downloads of things like operating system images), A different host.

Looking at my own domain, I have a shared server and www.mydomain.org maps onto a particular directory of my account on my shared area. My email maps onto a different subdirectory. Everything is virtualised now-a-days and actual individual physical machines are rare.

On a previous iteration of my setup I had subdomains e.g. tutor.mydomain.org which mapped onto yet another subdirectory - the host name being www.tutor.mydomain.org. On my current setup it actually maps onto a subdirectory of my "www" directory, so tutor.mydomain.org would be equivalent to www.mydomain.org/tutor - but in url terms my hosting company has it so they are equivalent! (So I saw little point in using sub-domains except as a short-hand)

I think most domains have redirect rules (see earlier regular expressions) to ensure that:

http://www.mydomain.org
https://www.mydomain.org
http://mydomain.org
https://mydomain.org

all map onto a single place on a server (and appear in the location bar in your browser to be one of the two https examples above). (I also "park" a 2nd domain so that mydomain.org and mydomain.org.uk are functionally the same - so that is 8 variations!)

https is the securer form of http (the hypertext transfer protocol) and is now widely adopted as the certification of the security has become much easier and cheaper.

Most servers will if you enter mydomain.org
assume: the https
redirect to a www or non-www form according to the rewrite rule
assume: a /
assume: a default page index.htm, index.html, index.php etc (by configuration)

The front page of the forum for instance is not fhug.org.uk/forum, but https:∕∕www.fhug.org.uk∕forum∕index.php - but the system tolerates the shorthand!

Where pages are created "on the fly" (as with KB and Forum pages) the server software may explicitly add some of the above, or may leave it to the server defaults. It will also in most cases account for users who do or don't use prefixes etc. when inputting links.

If your set up has an historic mix of hand crafted static pages and "served pages" (with multiple authors) it is quite possible that you will have a mix such as you have found.

The trick is to make the server tolerant of the variety.

[tatewise]

David, I am not sure whether your URL mappings are automatic or explicitly set up by the server administration.
Anyway, I don't see how it explains the following:

https://www.fhug.org.uk/ does NOT redirect to https://fhug.org.uk/ nor vice versa.
https://www.fhug.org.uk/forum/ does NOT redirect to https://fhug.org.uk/forum/ nor vice versa.

https://www.fhug.org.uk/kb/ does redirect to https://fhug.org.uk/kb/.
https://www.fhug.org.uk/wiki/ does redirect to https://fhug.org.uk/kb/.

[NickWalker]

Anyway, I don't see how it explains the following:

https://www.fhug.org.uk/ does NOT redirect to https://fhug.org.uk/ nor vice versa.
https://www.fhug.org.uk/forum/ does NOT redirect to https://fhug.org.uk/forum/ nor vice versa.

https://www.fhug.org.uk/kb/ does redirect to https://fhug.org.uk/kb/.
https://www.fhug.org.uk/wiki/ does redirect to https://fhug.org.uk/kb/.

I've only looked at this briefly but it looks like www.fhug.org.uk and fhug.org.uk are both pointing at the same folder on the server so one is an alias of the other. It doesn't matter which you use, you see the same thing.
The knowledge base is at location fhug.org.uk/kb but the www.fhug.org.uk/kb and www.fhug.org.uk/wiki don't point at that location and instead have redirects to fhug.org.uk/kb.

To answer some points Mike made in an earlier post that I've just seen... Some of the links on the FHUG home page are relative links so are in the HTML like this:

Code: Select all

<a href="https://www.fhug.org.uk/forum">Forums</a>,
<a href="/forum/ucp.php?mode=register">register for an account</a> to post to the forums.

The first link (an absolute fixed link) will always take you to the www.fhug.org.uk/forum URL whereas the second (relative link) will take you to the sub-folder /forum which will be www.fhug.org/forum or fhug.org.uk/forum depending on the URL you've used.

Once you get into the forum itself the links seem to be relative e.g.

Code: Select all

<a href="./viewtopic.php?f=38&amp;p=127227#p127227" title="Re: Stations of the British Army" 

so can be served from either www.fhug.org or fhug.org.uk depending on which you used.

[davidf]

Mike,

(Nick has partly beaten me to it!)

Jane and Helen for good reason probably won't discuss the exact server configuration, but on fhug.org.uk I can see at least three layers of configuration.

1. The default configuration provided by the host: https://krystal.uk/
2. The configuration of "the site" as decided by Jane
3. The configuration of the two main server programs running on the site: (1) Wordpress (customised and running with plug-ins) for the knowledge base and (2) phpBB (probably customised) for this forum.

I suspect mapping can happen at all three levels - and your "inconsistencies" may indicate that that is the case.

I don't think the inconsistencies matter because your examples end up serving the same information; it's just that the Wordpress application explicitly redirects.

(I seem to recall from when I ran a few websites at a University - in the last Millennium!) that "redirects" can be done two ways:

1. A sort of reinterpretation - you tell the server "when you see example.com serve up www.example.com"
2. A genuine redirect - you tell the server "when a request for example.com comes in forward it to www.example.com"

The difference is in response and amount of server activity - the former is in most cases more efficient.

[ColeValleyGirl]

A thought with regards to using the big search engines to search the FHUG site - Does FHUG have a sitemap?

Yes.

[ColeValleyGirl]

I suspect this discussion will benefit from some accurate information about the current state of search...

Why does Search matter so much?

The information contained within the FHUG Forums and the Knowledge Base is so complex and interlinked that it is impossible to design a navigation scheme that will lead to a specific piece of information every time, without relying on the person searching knowing 'FH terminology' and having a brain that works the same way as whoever designed the navigation scheme (or is at least experienced with the relevant navigation).

A search (''discovery') facility that:

• is fast
• is 'in-the-face'
• forgives' typos and 'imperfect' terminology
• suggests searches based on input so far plus searches previously done by other people within the FHUG/KB content
• allows search results to be narrowed down ('filtering' and 'faceting')

will make it much more likely that user find what they're looking for.

The Knowledge Base will live or die by its search capability. If newcomers can't find what they're looking for, they will not come back.

Search -- current technology and scope of searching

• Algolia search service. Searches KB; plus FH6, FH7 and AS Online Help. (Can be extended to other domains with permission).
  
  The KB search deliberately DOES NOT search the Forums because it should deliver 'fully-baked' information, not 'work in progress' or discussions. It also doesn't search 'external' links associated with KB contents (Related Content and Related Forum Discussions).
• FHUG:
  
  (1) Algolia (Quick Search top right) -- searches Forums and everything Algolia searches on the KB
  (2) phpBB search facility (Advanced Search and Search This Forum) -- searches Forums only.
  
  The Quick Search includes the KB because we'd love to avoid duplicate topics in the Forums, so if a fully-baked answer already exists, a search should point there wherever it originates.

Advantages of Algolia across both Forums and KB

• Using Algolia across all datasets has the advantage that the Forum 'Quick Search' and the KB search mechanisms will deliver consistent results although presented differently.
• Using Algolia also has the advantage that we only support two search technologies, and can deliver a consistent (as possible) user experience. Note: It would take a lot of effort to replace the phpBB search with Algolia and this option has not so far been investigated.
• Administering Algolia search behaviour is mostly done at the Algolia end, and the configuration is shared between the KB and the Forums. Configuration of results display is done at the KB/Forums end and depends on the relevant platform. KB display configuration is simple using a WordPress plugin and a bit of php. Forum display configuration is much less simple.
• Algolia functionality is addressed later.

History (for those who care)

• The Forums historically used phpBB search; the old KB (Wiki) used DokuWiki search. Both generated feedback (mostly the negative kind).
• In 2017, Jane introduced Algolia for the Quick Search in the Forums (covering forums and the old Wiki).
• In 2020, when we built the new KB, we looked at several options for implementing search capability. The default WordPress search is quite limited; there are many WordPress-specific search plugins available -- free and paid-for -- but none of them delivered the functionality that integrating the Algolia search service (via a free WordPress plugin) does.

What functionality does Algolia give us?

[Where Forums is mentioned, this only applies Forums Quick Search].

• Search terms can be words or part-words, or an exact phrase enclosed in double quotes "". You can also exclude any results containing a particular term by prefixing it with a - . [KB and Forums]
• If you enter more than one search term, the search will look for content containing one or more of those search terms, and return the results in order of relevance. Broadly speaking, the more terms that match, the higher up the list a result will appear. [KB and Forums]. For example, searching for 'restore backup' will first return items containing both the words restore and backup, and then return items containing either restore or backup.
• Tolerant search terms. Slight typos and variant spellings are accepted. If you enter a plural, the search will also look for the singular (and vice-versa). All search terms are case-insensitive. Terms will be found whether they’re capitalised or not. [KB and Forums]
• Synonyms. The search will take into account any synonyms that have been defined for words -- this means that an author can specify words that don't appear in an article but will still be searched. Algolia will also allow synonyms to be specified centrally, but we haven't exploited this yet -- lack of admin time! (There is a facility to allow Algolia to work out the synonyms, but it costs 50% more, so we haven't gone there). [KB only until we do it centrally].
• Autocomplete -- as you type, results are suggested, based on previous searches within the KB. [KB only]
• Faceting/filtering -- the ability to narrow search results down by 'metadata' (data outside the main article/post etc.)
  e.g Year posted [Forums]; or Content Source and/or Skill Level and/or FH version [KB]. Other filtering would be possible if worth the effort.
• Cross-domain searching -- we can search the FHUG forums, KB, online help files and potentially other domains with permission; and restrict which domain results we display in each context. [KB and Forums]
• Analytics -- to be honest, the admins haven't had the time to explore the capabilities here.

Why not use Google?

Google does not deliver:

• Synonyms
• Cross-domain searching
• Autocomplete
• Faceting/filtering

It can deliver an acceptable result for an experienced user, but not for a newcomer who doesn't know the terminology, nor can it signpost results outside the FHUG domain, nor filter search results based on data that Google can't search (e.g. FH version in the KB).

It's also worth noting that Google may (will!) prioritise sponsored content (ads to you and me); that their search result algorithm is opaque (and we can't tailor it); and that they are known to provide differing results based on your location and on previous searches that you've done. We're also very reluctant to turn Google Analytics on, for all sorts of reasons... including privacy and performance.

[jimlad68]

Helen, an excellent and informative post. And, together with other explanations in this topic has certainly enlightened me, (including using other search engines than Google!) and will help me use search engines more productively.

I brought up the Extended Search for Keywords with Google Site Search (not just google of course) for 2 reasons:
- In some situations, with the right parameters, it can give more honed results or let you see the wood from the trees.
- I thought greater use of google and similar in the right circumstances would mean fewer calls on Algolia, less cost, but probably not enough to worry about.

Via Chrome extension Search the current site (I think this has been mentioned before), Site Search is usually my first option, probably through habit because I use it for many sites as the quickest way to search, but not necessarily the best!