If I change the LLM do all the posts get retranslated?
I was using a LLM hosted on groq, it was about 95% done when I started getting errors. After investigating it I realized that groq had deprecated the model I was using. So I setup a new model in groq and then updated the model parameters in discourse.
I see that discourse is now using millions of tokens with the updated translation LLM every single day but the progress bar in the translations status page isn’t progressing at all (still stuck at 95%).
So I’m trying to figure out what happens when the LLM parameters change? Does it start the translations all over again from scratch? If so is there any way to prevent it from doing that and just picking up from where it left off?
It depends. For example, if you translate using a Gemini model and then switch to another, you’ll be creating a new word cache. In my experience, I prefer to maintain three routes: two with the same model, and I’ve now activated a Gemini Flash Lite for new content. So far, no issues.
Everyday the LLM is consuming millions of tokens for translations (it’s hitting the daily limit for the API) but there is 0 progress in translation stats (0 incremental translations). See screenshots.
Okay so I turned on ai translation debugging from the rails console from SiteSettings. ai_translation_verbose_logs = true and for the last 2 days I’m seeing this in the /logs error logs page.
DiscourseAi::Translation: Failed to translate topic 5898 to zh_TW: {“error”:{“message”:"Rate limit reached for model openai/gpt-oss-120b in organization org_01kccx1baz5sXXX service tier on_demand on tokens per day (TPD): Limit 200000, Used 193366, Requested 7514. Please try again in 6m20.16s.
The same topic is going on using up all the daily tokens limit. I can’t understand why one topic would get stuck in a loop?
