I’m wondering if, with recent advancements in LLMs, would it be possible to skim through a user’s posts and reading history to give them a Recommended feed?
This would be a great feature for our Q&A focussed use-case. We have many distinct categories, very busy users(colleagues) and lots of extremely useful information that they simply won’t see. A good Recommended list would help some of our users get ahead of the problems they are trying to solve.
I guess our ideal would be a list comprised of common, significant and hot bugs that people are talking about, related to the tools and conversations the user is already demontrating interest in.
As someone who has been using Discourse AI features, many before they were made public, I’m eager to learn more about your site. Understanding the specific details you’ve noted will enable me to make more informed recommendations for Discourse AI, taking into consideration the unique aspects of your platform.
Would love to but sadly it’s a big corporate internal instance
I can perhaps give you an idea of what we’re doing though.
Background
Essentially it’s like an internal Stackoverflow where we share company and industry specific issues like “how the heck do I configure the proxy for this?” or “I can’t find any results for Bug123 in Google, anyone else solved this?”
Our Categories are tool specific, such as:
Kubernetes
Gitlab
Buggy IBM stuff with the worst UI you’ve ever seen
How could a recommended list be useful?
Currently if I read/reply to a topic in the Gitlab Category which talks about integration with other tools, such as Kubernetes, I get no notifications about related topics. So if there is a very useful post in the Kubernetes Category, I may not see it till very late. I have to find it myself, or subscribe to the right notifications.
Most of our users will not have good notification settings:
They are under pressure to get results
They don’t spend enough time on the platform to get the best out of their notification settings.
A recommended list would provide potentially useful information, on demand, with a single click. Essentially, “based on your recent interests, here are some topics that might be helpful”. Could be built from the user’s search history, hot topics, common keywords in topics they participate in etc. So if they never visit the Buggy IBM Category, nothing from that Category will be recommended, unless it happens to touch upon something they already have a strong interest in, for example, Gitlab.
I provide a “daily buzz” AI summary at the top of this site which gets updated every 12 hours and draws from 2 days of Posts.
The solution is proprietary and not in a public plugin however, but it demonstrates the concept.
And yes, it is prone to errors!
I’ve been meaning to get it to link content … UPDATE: done, just needed GPT 4 Turbo and some prompt engineering. GPT 3.5 can’t cope with that level of direction.
Thank you, I hadn’t seen this. It might prove fruitful if things change for us, but unfortunately it doesn’t appear to meet our current restrictions(which I failed to mention earlier ) .
Restrictions
Can’t use chat
Can’t use PMs
Can’t rely on AI to generate responses, and silently provide them to our developers – we make safety critical software
These restrictions are essentially why a list of recommended topics would be a good solution for us: no content is generated, chat is not used, pms are not used.
This would be an interesting way to provide a daily summary. As you say, hyperlinks would definitley be a great feature. Are you think at all of customising it to the user’s distinct set of interests? What made this approach appealing instead of leaning on the existing daily digest?
I suspect this is the truly challenging part of this request, but also where a tonne of value lies.
Most content hosting platforms have some form of customised recommendation system, I would be surprised if Discourse didn’t head that way at some point too. It’s arguably more acheivable than for platforms like Spotify/Youtube as everything is text based, removing a layer of error between the translation of image/sound to object/text/concept.
Think more outside the box with the request in the first post. While you have many valid constraints that have to be meet, that doesn’t mean there are also potentially better solutions.
Side note about Related Topics which I can honestly say I have seen a few thousand of them in the last few days and noticed this
Something worth keeping an eye on.
In reviewing many post in Related Topics for an English site (OpenAI) starting to notice that topics in Spanish tend to be grouped together and suspect that if they were first translated to English each post would have a different vector and thus be clustered with other post.
List key stats like top users and counts of posts / topics
List 10 or so interesting new topics with a brief summary of activity that happened during the day
List 10 or so interesting older topics that got significant activity during the day
List a summary of activity by the @team group linking to topics they made inline.
Sourcing: ALWAYS Back statements with links to forum discussions.
Markdown Usage: Enhance readability with bold, italic, and > quotes and links.
Linking: Use https://meta.discourse.org/t/-/TOPIC_ID/POST_NUMBER for direct references.
User Mentions: Reference users with @USERNAME
Add many topic links: strive to link to at least 30 topics in the report. Topic Id is meaningless to end users if you need to throw in a link use ref or better still just embed it into the sentence
Categories and tags: use the format #TAG and #CATEGORY to denote tags and categories
bug topics are super important to me, make sure to prioritize them and also feel free to try to highlight any miscategorized bugs if you find any. DO NOT mention if stuff is correctly categorized.
When providing a topic summary, do so in paragraph form eg:
Experimental Admin Sidebar Navigation - @martin introduced an experimental admin sidebar navigation, sparking discussions about its design and functionality. Users like @packman and @Don provided feedback on missing entries and mobile display issues, which @martin acknowledged and is considering for future updates. Read more.
Recommended Topics List - @Tris20 suggested adding a list of recommended topics to the top menu, leveraging advancements in LLMs to tailor content to users’ interests. The conversation evolved with contributions from @EricGT, @merefield, and others, discussing the potential and challenges of personalized content recommendations. Read more.
Ten Years of Discourse - Users like @Judy_Hawkins and @Quercus shared their gratitude for various Discourse communities that have impacted their lives, celebrating the platform’s ten-year anniversary. Read more.
Locations Plugin - @Don reported an issue with the Locations plugin, which @merefield promptly responded to, indicating a willingness to investigate the problem. Read more.
Restricting Users from Editing Navigation Bar - @Yola sought help to restrict users from editing the navigation bar, and @JammyDodger clarified that the feature is designed for individual customization. @pfaffman and @LOCOSP provided CSS solutions to hide the edit button. Read more.
Upload File Bugs - @Vladimir_P encountered bugs when uploading files, which @JammyDodger and @blake addressed, leading to a fix being implemented. Read more.
Creating and Installing Plugins - @Gaurav_Kumar_Sandan inquired about creating and installing plugins on the home page, with @JammyDodger and @pfaffman providing guidance on theme components and plugin installation. Read more.
DiscoTOC - Automatic Table of Contents - Users like @digitaldominica and @Arkshine discussed issues with the DiscoTOC component, which were resolved following updates to Discourse. Read more.
Custom User Setting in a Plugin - @pirhoo sought advice on creating custom user fields from a plugin, with @merefield providing resources to assist in the process. Read more.
Billing Details Location - @Teresations needed help locating billing details for their Discourse hosted with Communiteq, which @awesomerobot and @RGJ assisted with, pointing to the control panel. Read more.
Interesting Older Topics with Significant Activity
Emoji Line-Height Adjustment Feedback - @sam and @tynaut discussed the impact of emoji line-height adjustments on user status styling, with @tynaut confirming a fix for chat and looking into a solution for posts/messages. Read more.
Using FILTER to Summarize Data - @JammyDodger shared a SQL query using the FILTER function to segment user data based on activity summary preferences, providing a detailed explanation of the query’s components. Read more.
Communication Style as Part of a User’s Profile - @Saif and @simon discussed the potential and concerns of implementing sentiment analysis on user profiles to assess communication style. Read more.
Enter to Newline, Shift+Enter to Submit Message in Chat - @meglio proposed a configuration change for chat input behavior, supported by users like @Jagster and @Moin, to make multi-paragraph messaging more user-friendly. Read more.
Can’t Approve a User’s First Post if It Has a Poll - @Firepup650 reported an issue where a user’s first post containing a poll could not be approved due to a permissions error. Read more.
Error Message When Uploading Logo - @spicerunner reported an error message when uploading a logo, which was resolved after an update. This topic was correctly categorized as a bug. Read more.
Outbound Notification E-mails Failing - @managenet faced issues with outbound notification emails after attempting to change the attachment size limit. Read more.
Can’t Rebuild App Stuck Extracting One Layer - @LOCOSP encountered a problem with a Docker layer extraction during a rebuild, seeking advice on how to force a re-pull of the files. Read more.
Does Discourse Launcher / Docker Use the Local Postgresql and Redis Servers? - @dalu74 inquired about whether Discourse uses the host’s Postgresql and Redis installations, which was clarified by @hello-smile6. Read more.
Specifically you could unleash some of these automation reports to try to surface interesting content.
The fact you control the context size, amount of days the report spans and more lets you have extreme control here.
Warning though, this needs GPT-4 turbo to work well, Anthropic Claude is a large context window llm but the results it has produced failed to impress me.
You may know this but also stating this for others as this may not be so obvious.
If you have access to the Discourse AI bot which can be found in the upper right corner
then using Forum Helper persona you can also try out different instructions that could be used with the custom instructions of the Periodic report using AI. In other words you can use Forum Helper to quickly prototype the report.
As one who was privileged enough to assist with feedback during development I can tell you that the custom instructions are important for getting the desired results. In the example given above I can clearly see that Sam added this for your variation of a report based on your initial request
I love the look of this, though I’m not sure I follow the functionality fully. It looks like this is producing something similar to the Digest. I quite like this new format for that information.
What I couldn’t see from the response, is whether or not this can provide a customised summary. Did I miss that somewhere? Is it possible for this to be generated per user, based on the user’s individual interests?
To make these thoughts more concrete, this is roughly what I had in mind:
Identify main interests of user: (Run once) Scan All topics and replies user has interacted with(like, reply, create) for keywords
Identify recent interests of user: Scan topics and replies the user has read this month for keywords
Identify key concepts of recent topics: scan all new topics this week
The piece that people love most that is LLM driven is the simple 1 paragraph recap of changes in a topic from date X to Y.
Instead of getting the LLM to read through the entire corpus and create a report like this we could just keep a “fragment cache” where say we store, paragraphs per topic:
topic id
date range for summary
very short summary
2343
12-1 → 12-9
…
With that we can construct custom summaries per user. So we are basically splitting this into multiple problems:
Identifying interesting topics → traditional programming (watched tags / categories or maybe “background” watched based on activity - exclude already read)
Recapping topics → LLM powered feature
This has multiple advantages. It scales very well, summaries can be much more accurate cause the amount of context is far more limited and there is less risk of context bleed and hallucination.
I’ve been contemplating a variation of this idea. Instead of thinking about topics related by tags or categories, consider them based on their semantic similarity. While not all semantically related topics will neatly form a sphere, it’s helpful to think about them that way. Imagine a sphere starting as a single topic; as more topics are added, the sphere expands, but it has a limited size after a while. Yet, more topics can be added within the sphere as it grows and over time even after reaching full size. Think of this like a fireworks display in the sky, with rapid explosions representing topics with different semantic similarities being added over time. This analogy helps me understand the concept better. I’m particularly interested in those rapid explosions and those that linger and twinkle for a long period. If one were to also have a visual representation of this it would be nice to pick semantic centers and tailor a paragraph about that.
This topic when using Show Full Post shows a real image of semantic related topics.
EDIT
Here is a visual that is closer to what I am thinking.
I have to be honest and note this is the first I have heard of “the algorithm” but yes it aligns quite a bit with what I am thinking. Will have to look further into it but I don’t expect that I would disagree with what you note.
Twitter’s algorithm refers to the complex set of mathematical rules and processes that the social media platform uses to determine what content is shown to users in their timelines and feeds. This algorithm is designed to curate and personalize the content that users see based on their interests, engagement history, and other factors. Twitter’s algorithm has evolved over the years, but it generally aims to show users the most relevant and engaging tweets while promoting user engagement and retention on the platform.
Key components and factors that may influence Twitter’s algorithm include:
Relevance: The algorithm considers the relevance of a tweet to a user’s interests and preferences. It takes into account the accounts a user follows, the content they engage with (likes, retweets, replies), and the keywords and hashtags they use in their tweets.
Engagement: Tweets that receive more likes, retweets, replies, and shares are more likely to be promoted by the algorithm. Higher engagement indicates that a tweet is resonating with users and is considered more valuable.
Recency: Twitter often prioritizes recent tweets to keep users updated with the latest information and trends. However, it may also show older tweets that are still relevant and engaging.
Personalization: The algorithm creates a personalized experience for each user by tailoring their feed to their specific interests and behaviors. It adapts over time as users interact with the platform.
Follower count: The number of followers an account has can also influence the visibility of its tweets. Accounts with more followers are generally more likely to have their tweets shown to a broader audience.
Media type: Tweets with images, videos, or other media tend to receive higher engagement and may be promoted by the algorithm.
Quality and safety: Twitter’s algorithm also aims to promote high-quality and safe content while reducing the visibility of spam, fake news, and harmful content.
It’s important to note that Twitter periodically updates and refines its algorithm to improve the user experience and address concerns related to the spread of misinformation and harmful content. Users may also have some control over their timeline by adjusting their settings, muting accounts, or using Twitter’s features like Lists and the “Show the best Tweets first” option to customize their feed further.
I absolutely love this. It’s feels like an alert-less version of Watching, which is exactly the kind of thing we are looking for. Something that really impresses me is how little cognitive effort I feel while reading this format! I honestly can’t wait to read more of them!
Am I right in thinking this has only been tried on Meta so far? I’m extremely curious about how it would handle a technical forum with potentially more obtuse information.
) .
