Detecting the kind of spam we see on our forum with an LLM is really like killing a fly with an atom bomb. The watched words small language model would pretty much be enough if only it was applied to edits.
I’m hesitant to use the LLM spam detection because I don’t know how much energy (and CO2) it uses (probably at least a million times more than the watched words feature), and it might have unexplainable false positives. The guide also makes it seem very difficult to set up (and I’m not an admin, so I’ll have to ask someone higher up to figure this out).
IMO, give a Decision 1 model a try. I use this workflow to isolate users that service account/group emails and have an auto-response to email notifications. I also added one for any solicitations that come to our community email address. I found a LLM to be a little too unreliable for this, but implementing a decision 1 agent has been bullet-proof.
Looks like workflows has a Post Edited option as well.
Here is the json with the post edit trigger added, if you want to try import it. You would just have to update the prompts to be for your specific scenarios.
Apparently the AI spam filter is enabled on our instance, but blatant spam like this still makes it through:
Appreciate everyone sharing their perspective on Life after Haskell. When I started exploring life a few weeks back, finding a straightforward option around life and haskell saved a ton of trial and error.
If anyone wants to compare, here is the site I settled on: [..].pages.dev/[..]
I have pages.dev in my watched words, but the second paragraph was added in an edit immediately after it was posted so it doesn’t go through the normal watched words. Watched words flagging does work as mentioned above, but with only one flag it is still visible publicly.
Edit: apparently the spam detection on this site is tuned quite aggressively
Edit 2: ah, it seems we didn’t activate the AI spam detection correctly. Now it does seem to work better. The decision model alone doesn’t seem good enough, but the LLM is able to detect it now. Thanks!
