Censor words should support sentence level censoring for Chinese

zogstrip · 2026 年2 月 16 日 14:45

我们现在应该可以正确处理中文的“词边界”了，这要归功于

github.com/discourse/discourse

FIX: support CJK and spaceless scripts in watched word boundaries (#37844)

main ← fix/watched-words-cjk-boundaries

opened 02:44PM - 16 Feb 26 UTC

ZogStriP

+84 -9

Watched words failed to match in CJK (Chinese, Japanese, Korean) and other space…less scripts because word boundary detection relied on whitespace or non-word characters. Languages like Chinese don't use spaces between words, so "测试" inside "这是一个测试文本" was never matched. Introduce a SPACELESS_SCRIPTS constant covering Han, Hiragana, Katakana, Hangul, Thai, Lao, Myanmar, Khmer, and Tibetan Unicode ranges. Update `match_word_regexp` for both Ruby and JS engines so that characters from these scripts are treated as word boundaries. This allows a CJK watched word to match when surrounded by other CJK characters, and a Latin watched word to match when adjacent to CJK text (e.g., "Test" in "我的Test很好"), while still preventing partial Latin matches (e.g., "Testing" does not match "Test"). Also fix the admin watched word testing modal to use `RegExp.exec()` with capture group extraction instead of `String.match()`, since the new boundary patterns include a leading consuming group. Remove the outdated "non-chrome browsers do not support lookbehind" comment — all major browsers have supported lookbehind since 2023. https://meta.discourse.org/t/71288 https://meta.discourse.org/t/396109

话题		回复	浏览量
Censored words do not respect word boundaries in non-latin alphabet Bug	8	1527	2018 年11 月 29 日
Censored pattern Bug	8	2243	2018 年1 月 12 日
* wildcards in Watched Words (Censor) don't work Feature	20	3178	2018 年1 月 11 日
Hope Watched words adds support for non-English characters Bug	2	47	2026 年2 月 20 日
A closing round bracket breaks word censoring Bug	5	1443	2017 年9 月 13 日

Censor words should support sentence level censoring for Chinese

相关话题