I checked the audit logs and it returned an empty response. I re-triggered the rule and it worked this time.
5개의 좋아요
Hi,
Not sure where to report this or if it’s worth reporting, however, it seems that the Discourse AI summary feature hallucinates - quite a few times.
Example:
A topic was created, there were no replies nor interactions from users. No solutions were discussed nor investigated. See post.
Yet, the bot confidently states “Some options were discussed but no definitive solution.”
Is there no way to filter this so that if a post does not have replies, the summary is not imagined?
1개의 좋아요
If you mean ‘can you get Bert to stop hallucinating’ then no, that is not something I can consistently do. 
I think the underlying tools get better all the time though, and accuracy improves along with them.
6개의 좋아요
I do plan to eventually move to amalgamation of topic summaries for the topic summaries section
It can eliminate a bunch of problems, but increasing reliability here with the current tools will certainly increase cost a lot, you would need to generate and then run rounds of validation, could get very costly
2개의 좋아요
On Daily Summary (5am UTC) - #149 and Daily Summary (5am UTC) - #148, it seems that some topics have been spelt incorrectly. Instead of #contribute:site-feedback, it was #site feedback, hence the topic did not get the link. Then, instead of #dev:translations, it said #contribute:translations. Sinilarly, it entered #Self-hosting instead of #documentation:self-hosting.
Yes, this is a known thing. It’s not really worth the effort to manually edit them or spend a lot of time tweaking the prompt to work around it. 
3개의 좋아요
Bert has been eating the wrong mushrooms again
5개의 좋아요
But… but you didn’t even post in that topic lol.
(Also, what Bert and mushrooms?)
Bert generally seems to be the nickname used for Meta’s AI bot.
1개의 좋아요
Seems like this was fixed? AND formatted the same way as the original few posts? (For this one, but seems like the 1pm one has the caregories fixed, though it doesn’t look like the first few (where caregories are bolded))
We moved to Claude 3.5 Sonnet so the quality is a bit better.
3개의 좋아요
Did AI just give up putting them (categories) in a class ‘mention’? Or was the prompt changed?
https://meta.discourse.org/t/daily-summary-5am-utc/291851/179?u=natedhaliwal
EDIT: Looks like it fixed itself in the next post.
It seems that AI did not group topics of the same category together, so the post has many many topics, with it in no apparent order.
(Not sure if this post belongs here)
https://meta.discourse.org/t/weekly-summary/291947/30?u=natedhaliwal
Almost correct
oh com’on. @Moin did the first one and the second topic was about something else lol
I’m flattered that bot thinks I’m doing Moin’s stuff tho. 
3개의 좋아요
No feature request was made as one already exists. I just linked it. This was in Support 
1개의 좋아요
I haven’t participated in that topic 
2개의 좋아요
oh now bot thinks I’m Hugh. 
3개의 좋아요
It made a hugh mistake
5개의 좋아요