Automatically add a space before the emoji if the user is typing Chinese and uses emoji picker

Contribute Feature

AI 生成的摘要

A discussion is ongoing about improving the Chinese user experience on Discourse forums. The issue is that emojis from the emoji picker appear without a space before them, causing confusion. snakeninny brought this to the attention of the Discourse team and codinghorror asked for input from joffreyjaffeux.

Sam mentioned that a partial fix was implemented, but a more comprehensive solution is needed. Schungx suggested always putting a space before the emoji code when selected from the picker, unless the preceding character is already a space.

Sam outlined four possible solutions:

  1. Allowing emojis to be rendered without punctuation, but this would be computationally expensive.
  2. Fixing autocomplete to behave consistently with the engine’s interpretation.
  3. Inserting a space prior to an emoji if needed after autocomplete.
  4. Adding a special mode to Markdown that requires no punctuation between emojis.

Sam suggests combining solutions 2 and 4. Tgxworld and misaka4e21 agree that a locale-based site setting would be the best approach. Misaka4e21 also points out two problems with the current implementation: the lack of a space before the first colon in emojis chosen from the picker, and the inconsistent behavior of auto-completion.

A solution is being implemented in PR #7429, which will insert a space before an emoji if there isn’t any, as confirmed by venarius.

1

Hi Discourse team:

I have been using Discourse for 4 years and love the modern design and quick update. Thank you!

Recently I have noticed some issues in my forum, that in China, we seldom use spaces like you do in English, so basically every emoji from emoji picker is screwed like this:

呵呵:thinking:

Which is annoying and confuses most Chinese users. Can you think of a way to improve the Chinese user experience?

Thank you!

Yinglu

7 个赞
Support emoji in CJK context without spaces
2

Any ideas @j.jaffeux?

4

We half fixed this with:

https://github.com/discourse/discourse/commit/d2a7a4916abecea389c99ae84de3bd40517b00d9

But what we need to do is match the implementation here:

https://github.com/discourse/discourse/blob/master/app/assets/javascripts/pretty-text/engines/discourse-markdown/emoji.js.es6#L154-L156

Which is checking for isSpace or isPunctChar.

3 个赞
5

Any progress? Got complaints from my users again today ;(

6

You can do this much simpler by always putting a space before the emoji code when selected from the picker.

You can also skip this space if the character immediately before is space.

1 个赞
7

Do you mean doing this manually or programmatically?

8

That will need Discourse core to be modified. Probably somebody can write a simple plugin to do this..

1 个赞
9

We have a few options here:

  1. Allow 呵:t_rex: so :t_rex: is an emoji. The problem though is that it is very expensive to detect Chinese per https://github.com/alsotang/is-chinese/blob/master/ischinese.js so this would add a bunch of extra slowness to the MD engine.

  2. Fix it so autocomplete behaves consistently with the way the engine interprets it. Meaning 呵:t should not pop open an emoji window.

  3. @schungx suggestion where after autocomplete is done we insert a space prior to emoji if needed.

  4. Add a “special mode” into Markdown where it requires no special punctuation between emoji and then a:t_rex:a would render a :t_rex: a.

I feel like the best way forward here is (2) and (4) combined. So:

  • Out-of-the-box autocomplete is not doing any funny business where it pretends to add an emoji and does not

  • In Chinese forums where words are just joined we would set SiteSetting.require_punct_before_emoji = false (and have autocomplete respect this)

@misaka4e21 / @tgxworld what are your thoughts here?

8 个赞
Emoji not appearing without space
10

Yea I think a locale based site setting is best way to go here.

3 个赞
11

There are two problems:

  • (a) If an emoji are chosen from the emoji picker, it will lack for a space before the first colon :, not only in Chinese posts, but in English posts as well. Users will get a not rendered a:smile: (a:smile:) if they input a and then select :smile: (:smile:) from the emoji picker.
  • (b) A colon : after a Latin character won’t cause auto-completion, yet a colon after a CJK Unihan character will. It could be solved exactly by (2).

To solve (a), (3) should be implemented in the emoji picker, not / not just in auto-completion.

(1) and (4) are in different logic.
To check if a character is in Chinese or not, we could try:

Though I don’t know whether the regular expression above will provide higher performance.

3 个赞
12

I remember going down the regular expression route in the past but that means we carry this giant regexp everywhere we go. For chinese forums, we can just have a site setting that does not require a space before : to render emojis.

5 个赞
14

@venarius is UX: Added leading whitespace to emoji picker & inline translation for… by venarius · Pull Request #7429 · discourse/discourse · GitHub going to fix this ?

5 个赞
15

Yes, this will insert a space before an emoji if there isn’t any.

5 个赞