The discussion revolves around a bug in Discourse’s search functionality for Thai language. Koft_Nattapol reports that searching for Thai words only works if the term appears at the start of a sentence or after a space, but fails when the word is in the middle — for example, searching “ไทย” returns no results, while “ตรง” or “เว้น” (at start or after space) work. They are evaluating Discourse for adoption in Thailand and seek a fix. zogstrip suggests enabling the search tokenize chinese japanese korean setting, but Koft_Nattapol confirms it doesn’t help. sujato explains that Thai differs from CJK languages: Thai syllables are formed by multiple characters (consonants, vowels, tone marks), often without spaces, making tokenization complex. They note that similar issues may affect other South/Southeast Asian languages using related scripts. siriwatknp adds that tag searches in Thai also behave inconsistently — e.g., tag “มาตรฐานและใบรับรอง” returns no results in search, yet posts appear when navigating directly to the tag page — using Discourse 2.6.0.beta1. The core issue is Thai-specific tokenization for search, which currently lacks proper support.
It seems that Thai language is one of a language that lack capability in searching.
If you are starting with Thai character, or space then Thai character, it can be search. But when the searching term is in the middle of a sentence, it won’t be found.
Sample.
ตรงนี้คือภาษาไทยนะ เว้นวรรคก็ยังเจอ
If I search with the starting keyword “ตรง” it will return the correct position. Also if I search with the word that begin after space “เว้น” It still can be found.
While if I type the middle word “ไทย” the problem happen, it return not found.
I would like to employ Discouse as a new forum technology to my country and looking for the patch to be done.
With CJK languages, each character is (more or less) a word. But in Thai several characters combine to produce a syllable, where the vowel may be before, after, above, or below, the consonant, and various tone markers may also be added. To make it even funner, words are usually (but not always) one syllable and there is usually no space between words. There must be standard libraries to tokenize these for search, I’m guessing.
Incidentally, most of the south and south-east Asian languages use related scripts, so if Thai has these problems we may find other languages do too.
Any progress on this issue? In my case, search by tag in Thai give the strange result, some tag give no result (ex. tags:มาตรฐานและใบรับรอง) but when I go to /tags/มาตรฐานและใบรับรอง I see the posts