Discourse-Translator++

[BCHK]

Translating more than 2 million characters per month requires a payment. You may pick from any of the available subscription offers.

Does anyone have experience with how this 2 million characters per month translates into real life experience in terms of number of posts (given an average post size of what... something like 200 characters or something, and given the typical forum environment.

I'm not sure how to characterize this - but since there is no "retranslation" with each viewing - I guess its just number of foreign language posts per month that get translated. 2 million characters equals 10,000 foreign language posts per month...

If we get 30,000 posts a month, we're perhaps OK as long as less than 30% of our users choose the translation option... (and as long as the average length of posts is less than 200 characters...)

[tobiaseigen]

I've now enabled this plugin on community.namati.org so we'll see how it shapes up. One point that has already emerged is that an important language for us, Burmese, is not yet available.

If we are already talking about letting members improve translations, perhaps we could also offer a UI for e.g. "Translation not available. Provide translation?" which pops up the text to translate from for translation.

I haven't even checked to see if Microsoft supports every language supported already by Discourse, but as discussed elsewhere I am interested in adding to this list of languages for users to choose from even if they are not (yet) supported by Discourse and then helping make sure the languages we care about are supported by Discourse.

[jesselperry]

I haven't even checked to see if Microsoft supports every language supported already by Discourse

Here: http://www.microsoft.com/en-us/translator/faq.aspx

[tobiaseigen]

Thanks!

And here's the list to compare it to, though it may be a bit out of date: https://meta.discourse.org/t/human-readable-interface-language-selection/29375/4?u=tobiaseigen

[tobiaseigen]

@tgxworld does html need to be rebuilt for all posts to have the translation link show up on the post menu? It doesn't seem to work except on new posts or when I select the admin wrench > rebuild html. That wasn't happening on kabissa when I tried it last week.

[tgxworld]

@tobiaseigen Opps... I just discovered something major. You'll need to remove the plugin for now. Otherwise, there are a bunch of post processing that isn't happening for your post. If you go to /sidekiq there should be a bunch of jobs that are failing.

I'll fix it asap and release a new version.

[tobiaseigen]

OK sounds good. Done. Thanks!

Is disabling it from Admin sufficient?

[tgxworld]

Unfortunately not. You'll need to uninstall it.

[tobiaseigen]

ok, well THAT I can't do from my phone on my way to bed. Trundling down to get my laptop...

[tgxworld]

In order to automatically detect the language to check whether we display the translate button or not, every single post is submitted once. Now that I think about it, this is going to cost alot of money but it'll make it less awesome

You'll have to factor the above cost in as well.

[strager]

Why not just translate the summary to determine the language? In general, the topics are a lot less characters than the full body.....

[gdpelican]

In Loomio we determine whether to display the translate button or not by looking at the locale of the current user vs the locale of the user who wrote the post, with an additional planned feature around allowing a post author to specify which language s/he's writing the post in.

So something like

post_locale = post.locale || post.author.locale || I18n.locale
reader_locale = current_user.locale || I18n.locale
show_translate_button if post_locale != reader_locale

Not a perfect solution of course, but could maybe save you from checking every post every time at the least.

[tgxworld]

The plugin could adopt a heuristic approach but I find that such approaches generate confusion for the end users. For instance, a user with en locale with a post "You should watch 進撃の巨人動画!" would not result in the translation button showing when it obviously should.

I'm all for reducing confusion so the tradeoff I would take is to add a site setting to turn off/on the automatic post language detection feature. It'll totally eliminate the "why is the translate button not showing for this post" problem.

That being said, there are still quite a number of optimization I could make to help reduce the costs in general. Right now, the entire cooked post is sent to Microsoft which means that all HTML tags, images URL etc are being submitted up for translation as well.

[codinghorror]

I would try to keep it simple -- some of the proposals in this topic seem awfully excessive to me.

[tobiaseigen]

I can see how this can become overly complicated. But allow me to brainstorm.

What if this plugin ignored the interface language preference of the logged in user and the post author, and instead allowed the user to select the language to translate into when they click the translate button? The ui for this could be similar to the chrome desktop notifications and the default Language preference would be saved.

The languages available would be determined by what the translation service offers, not by interface languages supported by discourse. Or maybe an admin setting.

Also, if a translation exists, maybe the translate button could be a different color (like the heart?) to indicate it has been translated already by somebody.

Down the road if there is interest and funding for it, Ui could be added to allow users to see all the languages a post has been translated into, manually translate and make improvements to existing translations.

I know that in my community there is need for Support for languages not supported by automatic translation, like Burmese.

[tgxworld]

Ok I've fixed the major issue with the plugin.

However, you'll need to be on the latest Discourse for it to work as I needed to submit a patch in order to resolve the issue.

Do try it out and let me know if there are anymore problems

[tobiaseigen]

I've updated both of my sites to latest and enabled this plugin again. Looks like all is working well so far. Will keep you posted!

Many thanks.

[tobiaseigen]

FYI I am getting errors frequently now in the error log regarding the too long strings being sent for translation, like this - they all look the same. Details below. This does not seem to be fatal and the plugin is working fine otherwise, but it would be nice to sort this out.

Info:

Job exception: Request URL Too Long

Request URL Too Long
HTTP Error 414. The request URL is too long.

Backtrace:

/var/www/discourse/plugins/discourse-translator/services/discourse_translator/microsoft.rb:119:in `result'
/var/www/discourse/plugins/discourse-translator/services/discourse_translator/microsoft.rb:76:in `detect'
/var/www/discourse/plugins/discourse-translator/plugin.rb:65:in `block in execute'
/var/www/discourse/lib/distributed_mutex.rb:21:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:5:in `synchronize'
/var/www/discourse/plugins/discourse-translator/plugin.rb:64:in `execute'
/var/www/discourse/app/jobs/base.rb:154:in `block (2 levels) in perform'

env:

hostname	forum.namati.org-app
process_id	[31865, 4000, 7697, 12852, 98, 102, 104, 96, 19007, 7678, 12640, 4213, 8220, 103, 2438]
application_version	[4f7140fb3228ac8177500fd253ce549dbd9dea32, 266f3591ce4f4b9acaebf5ad304d0d7f466e8e44, af7d51e9233527544dda2fe4254ef5cd87de0c22, fb4a40e628fab3489c145726706692f9d8332105, b817bebdff2b743f89e9867988a29290bfd6a28e, 06f616792d9cdaed36eda862d91439c7c09d5362, 50d2ba5f8e8ea2c934dc20b1fe9b03deedfa352e, 9483940244690ecc06763622bd9317b2a35d589a, 47e25648dfa200ac266a6311ba585827638d2bcc, 6b236d3c836f91d3c57227072320ed24f7a0a483]
current_db	default
current_hostname	community.namati.org
job	Jobs::DetectTranslation
problem_db	default
opts	
post_id	[4764, 2373, 4361, 1208, 4214, 1563, 5230, 364]
current_site_id	default

[tobiaseigen]

A feature request: display translated version of about, faq, terms of service and privacy policy, with link to original version.

[tgxworld]

Hmm that might not be a good use case using Discourse translator. Usually, FAQs are a wall of text which will easily exceed the character limits for the translation services.