# Censored words do not respect word boundaries in non-latin alphabet
**URL:** https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916
**Category:** Bug
**Created:** [27 november 2018 om 23:56 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916 "2018-11-27T23:56:03Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@meglio](https://meta.discourse.org/u/meglio)
#### Post date: [27 november 2018 om 23:56 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/1 "2018-11-27T23:56:03Z")
A censored word: `ебля` (f…ck)
A word in post text: `употреблять` (To Consume)
---
### Author:  [@Stephen](https://meta.discourse.org/u/Stephen)
#### Post date: [27 november 2018 om 23:57 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/2 "2018-11-27T23:57:45Z")
Does that apply to English stuff too? Would [Scunthorpe](https://en.wikipedia.org/wiki/Scunthorpe) be censored?
Edit: nope, doesn’t appear to happen with English:
**Update:** it does happen if you tick the box to treat watched words as regular expressions under Admin-\>Settings-\>Posting. That’s to be expected, right?
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [28 november 2018 om 00:15 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/3 "2018-11-28T00:15:03Z")
A word will be censored based on this logic:
> non-word character + censored word (which could include non-word characters) + non-word character
‘word character’ is currently defined by the regex `\w` metacharacter. Unfortunately this is simply “a-z A-Z 1-9 and \_”
As @Stephen points out, you can toggle the “watched words as regular expressions” setting, and then define your own regular expression however you want. It is very tricky for us to have a single regular expression which works perfectly for word boundaries across all languages.
---
### Author:  [@meglio](https://meta.discourse.org/u/meglio)
#### Post date: [28 november 2018 om 00:38 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/4 "2018-11-28T00:38:13Z")
Yet it worked somehow before now, and I just noticed censoring in action after updating Discourse to the latest. Did something change in the settings that changed the default behaviour?
---
### Author:  [@danekhollas](https://meta.discourse.org/u/danekhollas)
#### Post date: [28 november 2018 om 01:39 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/5 "2018-11-28T01:39:08Z")
You might want to look how it is done in the autolinkify theme, where I was dealing with the same exact problem. (essentially listing non-eord chars by hand, exactly because \w does not handle non-lati alphabet.
---
### Author:  [@meglio](https://meta.discourse.org/u/meglio)
#### Post date: [28 november 2018 om 02:02 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/6 "2018-11-28T02:02:42Z")
Possible solutions for word-boundaries in non-latin characters:
**Approach 1 - use sophisticated regex.**
A few of them are listed here:
>
**Approach 2 - Unicode Word Boundaries js library**
> **[4 Word Boundaries - UAX #29: Text Boundaries](http://unicode.org/reports/tr29/tr29-9.html#Word_Boundaries)**
>
> Word boundaries are used in a number of different contexts. The most familiar ones are selection (double-click mouse selection, or “move to next word” control-arrow keys), and “Whole Word Search” for search and replace. They are also used in database...
[https://github.com/wikimedia/unicodejs](https://github.com/wikimedia/unicodejs)
* * *
Will any of that work?
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [28 november 2018 om 12:00 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/7 "2018-11-28T12:00:45Z")
> [@meglio](#):
>
> Did something change
I changed the behaviour slightly back in October
[https://github.com/discourse/discourse/commit/3c2608d41c3e19c3037571b9102f73b743053fbc](https://github.com/discourse/discourse/commit/3c2608d41c3e19c3037571b9102f73b743053fbc)
> [@meglio](#):
>
> use sophisticated regex
If we censor ‘badword’, the regex you screenshotted wouldn’t be able to deal with something like parentheses:
`(badword)`
If there is a way to improve the behaviour without introducing another dependency, that would be great. The approach @danekhollas took in autolinkify is
```plaintext
let leftWordBoundary = "(\\s|[\\([{]|^)";
let rightWordBoundary = "([:.;,!?…\\]})]|\\s|$)";
```
I’m not a big fan of listing characters manually, but if it works it works 🤷♂️. I’ll put it on my list to test this approach. If anyone else fancies giving it a try in the meantime, the change would be to roughly the same places I changed in [this commit](https://github.com/discourse/discourse/commit/3c2608d41c3e19c3037571b9102f73b743053fbc). The important thing is that `pretty-text-test.js.es6` continues to pass.
---
### Author:  [@danekhollas](https://meta.discourse.org/u/danekhollas)
#### Post date: [28 november 2018 om 12:37 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/8 "2018-11-28T12:37:27Z")
Small note: I intentionally left out single and double quotes in those character lists which should be included in this use case.
---
### Author:  [@meglio](https://meta.discourse.org/u/meglio)
#### Post date: [29 november 2018 om 04:39 UTC](https://meta.discourse.org/t/censored-words-do-not-respect-word-boundaries-in-non-latin-alphabet/102916/9 "2018-11-29T04:39:01Z")
[offtopic] I really appreciate everyeone’s involvement and prompt replies in a discussion of such a minor issue. Could not express my appreciation by simply linking posts 🙂