Japanese translations are broken

Dev Translations
1

Some invalid unicode has been added to translations in Transifex to Japanese core/server.yml file.

new_user_of_the_month:
     name: "一ヶ月の新規ユーザー"
     description: "最初の一ヶ月間の功績\uD83D\uDC4C"

See the new_user_of_the_month.description.

How does this happen in Transifex, and how can we avoid it in the future? Any translators have insight on how Transifex can prevent this?

5 Likes
2

It seems that \uD83D\uDC4C is the UTF-16 encoding for the emoji 👌. (Charbase U+1F44C: OK HAND SIGN)

Similarly:

category:
    errors:
      invalid_email_in: "%{email}のメール認証に失敗しました\uD83D\uDE2D"

is supposed to be %{email}のメール認証に失敗しました😭

However, some other emojis seem to be exporting fine:

new_user_of_the_month:
      long_description: |
        ... おめでとうございます！🎁

I wonder if this is due to how the emoji is inputed into Transifex?

5 Likes
3

No Japanese translators have commented, and translations are very out of date, so we are considering removing Japanese translations soon.

3 Likes
4

I’ve been in contact with Transifex. It looks like they are doing some magic with Emojis that they shouldn’t do for YML files used by Rails.

– Whenever a Unicode sequence is included in a YML file, it detects this and represents this in Transifex Web Editor using the corresponding emoji symbol. That way there will be no need for translators to try to find what this code means.

– Whenever an emoji is added by a translator via Transifex Web Interface, our YML handler compiles the file, identifies the emojis that might be included in translations and then replace them with their corresponding Unicode representation in the generated YML files. Strings that contain such characters are enclosed in double quotes.

I hope they will stop doing this when the Rails YML handler is used. It doesn’t make sense, because the Psych YML parser supports Emojis, but doesn’t like the escaped Unicode characters.

A similar problem prevented the auto-updating of the core/client.yml file since the beginning of August. Their YML handler silently fails (well, it reports that it failed to fetch the file) when the YML file contains an Emoji. I fixed that on our side, until Transifex supports Emojis within YML files.

6 Likes
9

FWIW (possibly not much) we’re just about to launch a Japanese language community.

10

Well, the Japanese translations on Transifex still use Emojis. Unfortunately Transifex didn’t fix the root cause either, so the locale files haven’t been updated for quite some time.

You are welcome to contribute to the Japanese translations on Transifex if you need it for your community. We can start pulling translations again as soon as Transifex fixes the problem or translators remove the problematic Emojis.

5 Likes
11

Japanese translations are back. :tada:

No, Transifex didn’t stop putting Unicode in the locale files that isn’t supported by Ruby, but I added a workaround to our pull script that replaces the Unicode Surrogates with the actual Emojis.

https://github.com/discourse/translations-manager/commit/464e984c76220c9e45ab5d1c340e888c72437378

7 Likes