You might have noticed that short summaries have been enabled on Meta.
This will result in the topic list showing short summaries of the topics to help you get a quick overview.
Any feedback is welcome!
The gist icon is not showing on my site. I’ve set ai summary gists allowed groups to “everyone”. I’m running 3.4.0.beta3-dev. Did a hard refresh. Do I need ai bot github access token, and can you elaborate on what that does?
EDIT: I see How to enable compact vs expanded topic view now. So I guess you’re just looking for feedback on meta at this time.
The Documentation is here
Right now Gists are only available on Meta and a handful of sites. Once it is ready we will be rolling it out more widely. Will update the doc with more info as we are rethinking some of the settings
I have a request: can clicking on the gists button switch to the other type, without opening the model? Since there’s only one other option, it is time-consuming to choose the only other option.
Yeah, I believe we’ve heard this feedback before @awesomerobot. Certainly, an idea for us to consider
It might be useful to define some human generated benchmarks for the summaries, then keep iterating prompts until the results meet or exceed those benchmarks.
A couple of examples:
Gist for Understanding and managing bootstrap mode
Discourse’s bootstrap mode is a special state facilitating community growth, automatically adjusting user trust, communication frequency, and directory updates, and can be identified through the “Getting started” button or staff action logs.
Gist for a bug topic:
Forum theme is causing the user menu visibility issue, which can be tracked and solved, making it a manageable problem to fix despite being a little challenging.
I don’t think either of those excerpts are approaching human level performance - with human level performance defined as what can be achieved by a good writer. My guess is that the problem is that the LLM is being given an impossible task by being asked to fit too much information into a single sentence.
The primary goal of the excerpt should be to give users some idea of what to expect in the topic. It doesn’t need to do much beyond that.
It’s likely that the LLM that’s generating the excerpts needs to be aware of the context of the topic. For example, for the bootstrap documentation topic, I’d expect a simple definition of bootstrap mode. For a topic where the OP is a user generated question, the excerpt might just restate the question in terms that are likely to be understood by the site’s users. A topic initiated by a highly technical user might have an excerpt that uses a few technical terms, in order to attract the right audience to the topic.
Not really. Most of the time, a title accurately describes a topic’s content (that’s the desired goal, after all). If a title is too vague and can be improved, a T3+ or a staff user will often rewrite it.
But let’s elaborate a bit. I could see a better use of it with a different page presentation and a longer summary.
This 
, I think this is the goal. For all the reasons you mentioned it can be a difficult task for LLMs to go through the whole topic and understand the context → spit it out in 1-2 sentences
