GPT 3.5 turbo still not working for AI periodical reports

Contribute Bug

AI-generated summary

The discussion revolves around an issue with GPT-3.5-turbo not working for AI periodical reports in the Discourse platform. whitewaterdeu initially reports an error related to an unknown model, which is resolved by correcting the model name to gpt-3.5-turbo Roman. However, a new issue arises due to exceeding the 4097-token context limit, resulting in a context_length_exceeded error whitewaterdeu. Roman notes this should be automatically handled and suggests using gpt-4 as a workaround due to its larger context window. The thread is eventually closed as stale, with sam stating that GPT-3.5 should no longer be used, as newer models offer significantly larger context windows.

1

hello guys, i have updated my plugin , but GPT 3.5 turbo still not work

Message (3 copies reported)

Error running LLM report! : DiscourseAi::Completions::Llm::UNKNOWN_MODEL : DiscourseAi::Completions::Llm::UNKNOWN_MODEL

Backtrace

/var/www/discourse/plugins/discourse-ai/lib/completions/dialects/dialect.rb:27:in `dialect_for'
/var/www/discourse/plugins/discourse-ai/lib/completions/llm.rb:64:in `proxy'
/var/www/discourse/plugins/discourse-ai/lib/automation/report_runner.rb:67:in `initialize'
/var/www/discourse/plugins/discourse-ai/lib/automation/report_runner.rb:33:in `new'
/var/www/discourse/plugins/discourse-ai/lib/automation/report_runner.rb:33:in `run!'
/var/www/discourse/plugins/discourse-ai/discourse_automation/llm_report.rb:75:in `block (2 levels) in <main>'
/var/www/discourse/plugins/discourse-automation/app/models/discourse_automation/automation.rb:135:in `trigger!'
/var/www/discourse/plugins/discourse-automation/app/jobs/regular/discourse_automation_trigger.rb:13:in `execute'
/var/www/discourse/app/jobs/base.rb:297:in `block (2 levels) in perform'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:82:in `with_connection'
1 Like
2

Hey @whitewaterdeu :wave: - I suspect your automation script might still be using the wrong model name (gpt-3-5-turbo). Would you mind updating the model name in the script settings to gpt-3.5-turbo?

3

thanks :kissing_heart:, i have changed to gpt-3.5-turbo, and it worked, but it appeared another bug

it seems like i need to reduce tokens sent to openai, but i don’t know how to do it

Message

DiscourseAi::Completions::Endpoints::OpenAi: status: 400 - body: {
  "error": {
    "message": "This model's maximum context length is 4097 tokens. However, your messages resulted in 5605 tokens. Please reduce the length of the messages.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}


Backtrace

/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:91:in `block (2 levels) in perform_completion!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2353:in `block in transport_request'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http/response.rb:320:in `reading_body'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2352:in `transport_request'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:2306:in `request'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:19:in `block in request_with_mini_profiler'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/mini_profiler/profiling_methods.rb:50:in `step'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.0/lib/patches/net_patches.rb:18:in `request_with_mini_profiler'
/var/www/discourse/plugins/discourse-ai/lib/completions/endpoints/base.rb:89:in `block in perform_completion!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-http-0.4.1/lib/net/http.rb:1570:in `start'
4

You can’t do that. We should automatically trim content to make it fit the context window. Must be a bug in our code.

While we sort this issue, you could switch to gpt-4 which has double the context window.

3 Likes
6

closing as stale, GPT-3.5 should not be used anymore, vast majority of models now ship with very large context window.