What happens to translations when LLM changes?

Support
1

If I change the LLM do all the posts get retranslated?

I was using a LLM hosted on groq, it was about 95% done when I started getting errors. After investigating it I realized that groq had deprecated the model I was using. So I setup a new model in groq and then updated the model parameters in discourse.

I see that discourse is now using millions of tokens with the updated translation LLM every single day but the progress bar in the translations status page isn’t progressing at all (still stuck at 95%).

So I’m trying to figure out what happens when the LLM parameters change? Does it start the translations all over again from scratch? If so is there any way to prevent it from doing that and just picking up from where it left off?

2

It depends. For example, if you translate using a Gemini model and then switch to another, you’ll be creating a new word cache. In my experience, I prefer to maintain three routes: two with the same model, and I’ve now activated a Gemini Flash Lite for new content. So far, no issues.

3

Old translations are kept, new LLM will only do translations for what wasn’t translated so far.

4

What can I do to verify what’s going on?

Everyday the LLM is consuming millions of tokens for translations (it’s hitting the daily limit for the API) but there is 0 progress in translation stats (0 incremental translations). See screenshots.

IMG2422
IMG24221170×1824 111 KB

IMG2420
IMG24201170×1762 173 KB

5

Enable verbose logs and check the ai API audit logs table last entries.

6

Okay so I turned on ai translation debugging from the rails console from SiteSettings. ai_translation_verbose_logs = true and for the last 2 days I’m seeing this in the /logs error logs page.

DiscourseAi::Translation: Failed to translate topic 5898 to zh_TW: {“error”:{“message”:"Rate limit reached for model openai/gpt-oss-120b in organization org_01kccx1baz5sXXX service tier on_demand on tokens per day (TPD): Limit 200000, Used 193366, Requested 7514. Please try again in 6m20.16s.

The same topic is going on using up all the daily tokens limit. I can’t understand why one topic would get stuck in a loop?

7

Did you disable thinking for this LLM? Try that.

8

When you say thinking, do you mean vision? I don’t see any check box or option for thinking on the LLM configuration page in Discourse

Curious what impact would that have on translations ?

It’s been stuck on this page for the past week now consuming all the daily tokens:

Embedded Subtitles with MP4 Unprocessed - General Support / Questions - MCEBuddy

DiscourseAi::Translation: Failed to translate topic 5898 to fr: {“error”:{“message”:"Rate limit reached for model openai/gpt-oss-120b in organization org_01kccx1be8fffaz5sbe17 service tier on_demand on tokens per day (TPD): Limit 200000, Used 197080, Requested 7512. Please try again in 33m3.744s.

9

Do you mean reasoning effort or vision ?

IMG2438
IMG24381170×2089 215 KB

IMG2439
IMG24391170×1815 125 KB

10

@Falco if I set the reasoning effort to none I get this error:

DiscourseAi::Translation: Failed to translate topic 5898 to fr: {“error”:{“message”:“reasoning_effort must be one of low, medium, or high”,“type”:“invalid_request_error”}}

What setting are you referring to ?

11

So it’s been about 24 hours now, I don’t see any translations happenings in the logs, no errors, no successes for the last 24 hours.

However the dashboard is showing that the translations are incomplete.

image
image1423×777 52.3 KB

I’ve tried to disable and renable the automatic translations toggle but that didn’t make any difference.

When I look at the sidekiq scheduler, I only see this one task related to translations:
image

I tried to trigger it manually but there was no progress in the translations and no errors in the logs either.

Why would translations get stuck?

12

From what I remember debugging a similar issue for a customer, you simply can’t use a thinking model for locale detection.

We don’t use structured outputs there, which means we are very sensitive to how the output comes. Thinking blocks break those 100% of the time, and without locale detection translation is always stuck.

13

So if I understand this correctly, we will need to use non thinking model for AI translations.