AI Summary Backfill is summarizing PMs as well as proper topics. In PMs, there is no summarize button so there’s no way to ever see the summary. Either need to not summarize PMs, or add a button to PMs. I vote for not summarizing PMs.
We appear to have many more PMs than proper topics, so this is a big inefficiency.
EDIT: I have ai pm summarization allowed groups set to everyone. Does this affect backfilling? Or just who is allowed to push the button?
As you found yourself, there is the button if you are in the allowed groups setting.
As we plan to allow showing the short summaries in the PM list, and to use both summaries to help our upcoming AI search feature, those are needed for several reasons.
That said, we can introduce a setting to skip PMs here, the same way we did for embeddings.
Very interesting dilemma…. I predict that in my community in the near future, the topic/PM split will be 50/50. I can see why for some communities it would be highly uneconomical to include PMs in anything AI related and why in others it would be invaluable. Perhaps it would be ideal to have a method by which discourse admins could decide from the very beginning whether or not to include PMs? This way, there can be an executive decision made based on the community type because sometimes AI has no role to play when it comes to basic question/answer PMs, and other times it may be critical if the community shares searchable/valuable information via PM.
Nonetheless, search is key here: does your community type benefit from generating embedding for PMs or not? In the end, it requires an ultimate decision I think
Our ratio is 10 PMs for every normal topic:
| topic_type | total_count |
|---|---|
| Private Messages | 52130 |
| Normal (Regular) Topics | 5399 |
(Data Explorer code is courtesy of ChatGPT o1.)
The normal topics often use somewhat more input tokens than PMs, which mitigates the issue somewhat, but still.