We have many topics that exceed typical context lengths (say 128k) and bad things happen. Have you considered summarizing based on only the most-liked posts? Start with the most-liked post, add the next-most-liked, and continue until the target context length is reached, then pass the resulting text to the GPT? For our forum at least, “likes” are a very good proxy for “most relevant”.
AI-generated summary
In a support discussion titled “Why Does My Forum Summarisation Keep Hanging?”, markschmucker reports that summarization hangs on long topics, suspecting it’s due to exceeding context length limits. Falco clarifies that for long topics, Discourse already uses the top 50 most-liked posts to stay within context limits — a strategy markschmucker supports, noting likes correlate well with relevance. markschmucker then shares an error log indicating their OpenAI account hit a TPM (tokens per minute) limit of 30,000 while requesting 45,906 tokens using gpt-4o. They note increasing limits requires spending $50 and waiting ~month+7 days, and find no way to cap tokens in Discourse. Falco suggests checking the ai_api_audit_logs table and updating Discourse to fix frontend bugs. Falco further explains the issue stems from a restricted OpenAI account and recommends switching to alternative AI providers like Anthropic, AWS, Azure, Google Cloud, or Llama API, which may offer higher free-tier limits. markschmucker confirms switching to gpt-4o-mini resolved the issue, noting OpenAI applies different TPM limits per model.
That is how it works on long topics, we use the top 50 most liked posts in the topic.
Hmm, summarizing appears to hang on long topics, so I assumed it was related to the content length. Any tips for debugging this? I’m using gpt-o4. I don’t see a way to view specific API requests in my OpenAI dashboard.
You can see requests on the ai_api_audit_logs table.
Regarding the hanging mid summarization, we had some front end bugs on that front recently, so please update your site to latest.
I found this in the error logs:
Request too large for gpt-4o in organization org-xxxxxxxxxxxxxxxxx on tokens per min (TPM): Limit 30000, Requested 45906.
To increase my limits, I believe I have to actually spend $50 (not just budget it) and wait something like a month plus 7 days:
So far I’ve only spent $2.00 and no payment is due yet. I also don’t see how to cap the tokens in Discourse. So I don’t see how to use summarization for long topics at this point.
Ohhh so the issue is that you have a restricted OpenAI account that limits you on tokens per minute, which impact using AI summarization on Discourse.
That’s is such a bummer. I’d look into alternative providers, like Antrhopic, AWS, Azure, Google Cloud or even Llama API providers. I believe there are providers with a free tier that will give you a higher limit than that.
I switched from using 4o to 4o-mini for summarization, and that appears to work on my longest topics. Apparently OpenAI has different limits for the different GPTs.