Watched words upload should support UTF8

when I upload a file(UTF8) with Chinese character, it were garbled . is this a Bug?
My Discourse version is : 2.3.0.beta3

Just confirming this is working fine if you use the UX, but has trouble when you upload a file?

3 Likes

from the following code, it seems only file encoded with ISO-8859-1 is supported to be uploaded.

 Scheduler::Defer.later("Upload watched words") do
      begin
        File.open(file.tempfile, encoding: "ISO-8859-1").each_line do |line|
          WatchedWord.create_or_update_word(word: line, action_key: action_key) unless line.empty?
        end
        data = { url: '/ok' }
      rescue => e
        data = failed_json.merge(errors: [e.message])
      end
      MessageBus.publish("/uploads/csv", data.as_json, client_ids: [params[:client_id]])
    end

    render json: success_json
2 Likes

@neil, do you remember why you enforced that encoding?

1 Like

Hmm no I don’t. Probably copy-pasted from somewhere else.

UTF-8 would be a more common encoding.

3 Likes

Fixed:

https://github.com/discourse/discourse/commit/1812a38f0a4194eba799ca56b492986db0088498

9 Likes

@Kevin_Sun can you update to latest and try again? You should now be able to import watched words with Chinese characters.

5 Likes

@zogstrip yes, after upgrade to latest, it works, thank you so much

4 Likes

This topic was automatically closed after 33 hours. New replies are no longer allowed.