Discourse has the ability to summarise long threads. My understanding is that it uses engagement metrics like reactions or number of replies to identify what posts should be kept. In my community (internal communication within the company) this almost never triggers despite threads that are 20 pages long (reply count is not the best trigger for us)
A more powerful approach would be to use an LLM like GPT-3 or BERT to summarise the whole thread on demand (vs just filtering the posts). Summarisation is a use case where LLMs shine (pretty much the poster child) and I believe this would be way more effective than the current approach which is based on weak signals (in some communities, people may not use reactions or reply to specific posts).
On the other hand, running GPT to summarise text can be expensive, up to $0.02 per 1k tokens. This feature would probably make more sense in some private communities than high-traffic public ones.
I imagine the feature to work like this:
when reading a long thread for the first time (long defined with a setting) that has never been summarised, suggest to summarise it on demand.
the summarisation would be inserted in the thread (eg: everything before is summarized)
it is linked from the top of the thread (eg: jump to the summary)
ideally, there is some feedback mechanism
after N more replies, offer to publish a new summary and update the link at the top.
the plugin would be configured with an openai key (assuming this is the LLM used). Eventually, if it proves successful, you would want to offer access to different models.
At least that’s the theory . In practice there are unknowns about how good it would be, or how to configure the degree of summarization. My instinct is that this is extremely high ROI for some communities. The number one problem I hear is that people hate catching up on threads. This would be of massive help.
import openai
import requests
from bs4 import BeautifulSoup
openai.api_key = "KEY"
url = "https://meta.discourse.org/t/summarising-topics-with-an-llm-gpt-bert/254951.json"
response = requests.get(url)
data = response.json()
messages = []
messages.append("Title: " + data["title"])
for post in data['post_stream']['posts']:
soup = BeautifulSoup(post['cooked'], 'html.parser')
messages.append("Post #" + str(post["post_number"]) + " by " + post["username"])
messages.append(soup.get_text())
text_blob = "\n".join(messages)
print(text_blob)
max_chunk_len = 4000 # Maximum length of each chunk
chunks = [text_blob[i:i+max_chunk_len] for i in range(0, len(text_blob), max_chunk_len)]
summaries = []
for chunk in chunks:
print("processing chunk")
response = openai.Completion.create(
engine="text-davinci-003",
prompt="prompt:\n" + chunk + "\nsummary:",
temperature=0.7,
max_tokens=200,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)
summary = response.choices[0].text.strip()
print("\nSUMMARY")
print(summary)
print("\nSUMMARY")
Discourse is exploring the possibility of using an LLM such as GPT-3 or BERT to summarize long threads on demand. This feature would be configured with an openai key and could be especially useful for private communities. The summarization would be inserted in the thread, with a link at the top and a feedback mechanism. The AI team is currently exploring this possibility and will update when they have something to show.
@Julienlavigne I’ve made some significant improvements to this in terms of stability, flexibility and administrative feedback.
Errors returned by the model are now logged, for example, so you can now easily tell if you’ve breached model limits.
If you (or anyone) wants to give it a try I might be able to provide a little casual support for it for a little while and may be able to incorporate feedback where reasonable.
Sadly I’m using the business version of discourse and can’t deploy custom plugins, so I can’t test this easily. I may try to run discourse locally with a backup and play with it though. I’ll keep you posted.
Thanks @merefield Robert, and yes I had seen that, and thanks for your work on this. I was hoping, however, that @sam (or someone with Discourse support) was working on an LLM summarization feature with full integration with native Discourse, Obviously, if Discourse business plan customers simply need to be charged a bit extra for their own individual OpenAI(or whatever the provider turns out to be) key, that would be much simpler from a configuration, uplevel compatibility and maintenance perspective. All the best.
I can’t speak for CDCK here but the nuance here is that I suspect many sites prefer this to be optional as some are nervous about AI (whilst others wish to embrace it fully).
So I suspect this will remain an optional plugin and allows CDCK to customise the product offering as they and customers see fit.
However, speaking as an engineer, that pretty much allows for a fully native experience and would not present any meaningful limitation from a functional perspective.
This is now available on your install, you can go ahead and plug in creds for anthropic or openai.
Roadmap wise, we are thinking of unlocking “embedding” next, which will give you semantic related topics. Following that I think the next 2 would be ai helper and summary.
I can’t put an exact date on when all these features will unlock on the business plan, but we intend to roll them out over the next few months.
Self hosters and enterprise customers can already access them today.
Summarization is particularly interesting, we are buying a few A100 servers and should have them racked up in the next few weeks. Mistral is surprisingly capable and fast, once we have the capacity rolled out we may be able to provide summary even to business plans without byo keys, but we need to first explore perf characteristics prior to making any commitments there.
feedback mechanism
. In practice there are unknowns about how good it would be, or how to configure the degree of summarization. My instinct is that this is extremely high ROI for some communities. The number one problem I hear is that people hate catching up on threads. This would be of massive help.
