A discussion is taking place about an issue with searching by Thai tags on Discourse version 2.6.0.beta1. siriwatknp reports that searching with Thai characters without extra vowels yields results, but searching with extra vowels does not. neounix suggests that the search engine may have problems with Thai vowels, tone marks, and diacritics.
siriwatknp asks for a workaround, and neounix recommends researching similar issues and examining the Discourse code. neounix also suggests trying other Thai character sets in the browser keyboard settings.
tgxworld offers to reproduce the problem locally and requests the text and search term. However, tgxworld later discovers that siriwatknp has submitted a pull request (PR) to fix the issue, which has been merged. The PR can be found on GitHub.
discourse version 2.6.0.beta1
These are example posts that have tag in Thai.
When I go to search page and try to search with Thai character (no extra vowel) the results match the amount from tag filter.
However, when I search with extra vowel, there is no result found. (but in tag filter there are 17 posts)
Hi K. @siriwatknp and Sawatdee from Thailand,
I am guessing the app search engine has problems with almost all Thai vowels, (4) tone marks and (5) diacritics?
Reference:
Any suggestion about workaround for this issue?
My first suggestion is to search the net for similar documented issues and try to find “the way” other search engines manage this layer of complexity; and then look at the Discourse code and see what changes might be required to improve this search algo.
OBTW: Have you tried other Thai char sets in your browser keyboard settings?
Note:
A quick search, I see some experts have proposed the “Two-Pass Search Algorithm” approach:
https://www.cicling.org/2008/RCS-vol-33/07-Kruengkrai.pdf
6 Conclusion
We have presented a discriminative learning approach for Thai morphological
analysis. We consider Thai morphological analysis as a search problem. We propose the two-pass search algorithm that finds the most likely path in the expanded search space. The objective of our algorithm is to increase the coverage
of word hypotheses based on probability estimation in the lattice. The experimental results on ORCHID corpus show that the two-pass search algorithm can
improve the performance over the standard search approach
See also: Computers and the Thai Language
https://lexitron.nectec.or.th/KM_HL5001/file_HL5001/Paper/Inter%20Journal/krrn_52085.pdf
This article explains the history of Thai language development for
computers, examining such factors as the language, script, and
writing system, among others. The article also analyzes characteristics
of Thai characters and I/O methods, and addresses key issues involved
in Thai text processing. Finally, the article reports on language
processing research and provides detailed information on Thai
language resources.
@siriwatknp Can you provide me with the text and the search term in text so that I can try to reproduced the problem locally?
@siriwatknp Just saw that you submitted a PR to fix this issue 
The PR looks good to me and has been merged.
