Posts too long for PostSentimentAnalysis

AI-generated summary

A user, Overgrow, reports encountering Net::HTTPBadResponse errors in their error log after enabling sentiment analysis using OpenAI, which appears to be linked to unusually long posts causing OpenAI API to return a 400 Bad Request. Despite adjusting the token limit to 64,000 (50% of the 128K context window), the issue persists. Jagster clarifies that the error is due to an incorrect HTTP status code and questions the setup process. Overgrow explains the configuration involves enabling the ai sentiment enabled module and entering LLM model parameters. Sam requests a screenshot of the configuration, which Overgrow provides. Falco corrects the misconception, stating that there is no sentiment analysis feature via OpenAI in DiscourseAI and directs users to the official guide for local setup.

After enabling sentiment analysis using OpenAI, my error log is filled with

Job exception: Net::HTTPBadResponse

Then in Sidekiq:

Jobs::PostSentimentAnalysis
Jobs::HandledExceptionWrapper: Wrapped Net::HTTPBadResponse: Net::HTTPBadResponse

Looking at the post_ids in question, it seems the issue is likely caused by the unusually long posts. Otherwise, the sentiment graphs generate without any problems.

I didn’t know the sentiment analysis can be used thru OpenAI :thinking:

To add more information, I’ve got Number of tokens for the prompt set according to the recommendation (64000 which is 50% of 128K context window). But I’m not sure if this plays any role..

No it doesn’t, because then you would get different error. What you get is exacly what it sound, wrong status code.

How did you set yp sentinent analysis using GPT of OpenAI?

The error message isn’t exactly descriptive. I don’t even know the exact HTTP code.

The culprit is clear, it has something to do with a post length. In that case OpenAI API returns 400 Bad Request.

You just enable the module ai sentiment enabled and it works as long as you have LLM model parameters entered.

I’ve experimented with setting as low as 10,000 tokens and no improvement. Net::HTTPBadResponse is still filling my error log.

Can you share a screenshot of your exact config, and your llm config.

Sure, here are the settings


There is no sentiment analysis feature via OpenAI in DiscourseAI. Please follow Discourse AI - Self-Hosted Guide if you want to run it locally.