# Khmer Unicode Zero Width Space disappears after posting

**URL:** https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260
**Category:** Bug
**Created:** [March 12, 2015, 12:25pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260 "2015-03-12T12:25:05Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 12, 2015, 12:25pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/1 "2015-03-12T12:25:05Z")

</div>

Hi there, I just installed discourse and found out that it didn’t support Zero Width Space at all. It is Khmer Unicode special character that represents a hidden space to separate each word. you can take a look here [http://www.askcambodia.org/t/angelababy/21](http://www.askcambodia.org/t/angelababy/21)

---

<div class="post-metadata">

### Author: ![supermathie](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/supermathie/32/507518_2.png) [@supermathie](https://meta.discourse.org/u/supermathie)
#### Post date: [March 12, 2015, 2:01pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/2 "2015-03-12T14:01:03Z")

</div>

Can you give us an example of how it should look?

Not being familiar with Khmer it’s hard to tell what’s going wrong.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [March 12, 2015, 9:56pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/3 "2015-03-12T21:56:52Z")

</div>

It should work, all unicode is supported. Where are you trying to use this character, specifically?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 12, 2015, 10:27pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/4 "2015-03-12T22:27:04Z")

</div>

unless … this is another nokogiri bug …

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [March 12, 2015, 11:10pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/5 "2015-03-12T23:10:46Z")

</div>

The OP did not say “entity”, and what was referred to is a Unicode character, not a string representing a HTML entity… so unless it is a strange HTML entity, it should work.

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 13, 2015, 2:06am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/6 "2015-03-13T02:06:11Z")

</div>

@supermathie the thing is that it replaces zero width space to be a space for example here

ប្រទេស កម្ពុជា this one with zero width space  
ប្រទេសកម្ពុជា this one with no zero width space

Thanks,

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 14, 2015, 4:43pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/7 "2015-03-14T16:43:30Z")

</div>

@codinghorror something makes the zero width space becoming a space. do you have any idea?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 14, 2015, 8:36pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/8 "2015-03-14T20:36:23Z")

</div>

It is a bug, we will try to find it, in the mean time perhaps you should look at translating Discourse 🙂

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 15, 2015, 6:33am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/9 "2015-03-15T06:33:34Z")

</div>

Hi Sam I just requested for khmer language on Transifex. hope it is approved soon and i can invite my team to work on that.

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [March 15, 2015, 9:53pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/10 "2015-03-15T21:53:43Z")

</div>

Just making sure @techAPJ sees this ^^

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [March 16, 2015, 2:58am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/11 "2015-03-16T02:58:41Z")

</div>

Approved! Thanks for contributing translations.

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 16, 2015, 3:13am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/12 "2015-03-16T03:13:14Z")

</div>

I am not seeing this bug in our markdown cooking code:

```plaintext

 it 'does not strip zero width spaces' do
    from = "hi\u{200B}\u{FEFF}there"
    cooked = PrettyText.cook(from)

    cooked = cooked.gsub(/<[^>]*>/, "")
    expect(cooked).to eq from
end

```

So let me take this one step back, does the preview look right before you post?

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 16, 2015, 8:30am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/13 "2015-03-16T08:30:08Z")

</div>

@sam Yes sure it works fine in the preview. It just breaks the words after posted.

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 26, 2015, 12:52pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/14 "2015-03-26T12:52:13Z")

</div>

@sam Any update on this?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 26, 2015, 10:14pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/15 "2015-03-26T22:14:36Z")

</div>

I am confused about the repro, in your example:

ប្រទេស កម្ពុជា

has a real space in the raw markdown, I need an example I can work with.

---

<div class="post-metadata">

### Author: ![lidel](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/lidel/32/115991_2.png) [@lidel](https://meta.discourse.org/u/lidel)
#### Post date: [March 26, 2015, 10:45pm UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/16 "2015-03-26T22:45:32Z")

</div>

@sam, I was able to replicate the issue this way:

I copied sample from:

> **[Zero-width space](https://en.wikipedia.org/wiki/Zero-width_space)**
>
> The zero-width space (rendered: .mw-parser-output .mw-tmpl-kbd{background:#EEE;color:var(--color-base)}@media screen{html.skin-theme-clientpref-night .mw-parser-output .mw-tmpl-kbd{background:#171a1d}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .mw-tmpl-kbd{background:#171a1d}}​; HTML entity: &ZeroWidthSpace; or &#8203;), abbreviated ZWSP, is a non-printing character used in computerized typesetting to indicate where the word boundaries are, witho...

and pasted here:

> Word Word Word

After each _Word_ you should find one zero-width space (U+200B).

Discourse displayed ‘WordWordWord’ correctly in preview pane (separated with U+200B), but after I posted this, I saw that it got cooked/refreshed and this operation replaced ‘WordWordWord’ with ‘Word Word Word’.

So indeed, Discourse replaces U+200B with real space ☹

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 27, 2015, 12:30am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/17 "2015-03-27T00:30:19Z")

</div>

@lidel thank you very much for the explaination. that is the point @sam. 🙂

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 27, 2015, 2:03am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/18 "2015-03-27T02:03:54Z")

</div>

AHA I can see why this is happening

@zogstrip added this …

[https://github.com/discourse/discourse/commit/f4208ae83fd43e0cdd663d82a73fabfb65f327bb](https://github.com/discourse/discourse/commit/f4208ae83fd43e0cdd663d82a73fabfb65f327bb)

This is actually a case of us being too smart for our own good. It was added as a system to avoid people “gaming” edits and bypassing rules. However it is totally not really acceptable that we break formatting here, so I patched it with:

[https://github.com/discourse/discourse/commit/58c95f64d2887134604b8024afb0d673c48f433f](https://github.com/discourse/discourse/commit/58c95f64d2887134604b8024afb0d673c48f433f)

---

<div class="post-metadata">

### Author: ![evannak](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/evannak/32/115773_2.png) [@evannak](https://meta.discourse.org/u/evannak)
#### Post date: [March 27, 2015, 2:06am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/21 "2015-03-27T02:06:26Z")

</div>

@sam so how can i fix it? or wait for another release of discourse?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 27, 2015, 2:07am UTC](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260/22 "2015-03-27T02:07:44Z")

</div>

Wait for tests passed to pass and then update your instance if you are on the tests-passed branch which is the default in our configs.

[Next page](https://meta.discourse.org/t/khmer-unicode-zero-width-space-disappears-after-posting/26260.md?page=2)
