AI Triage has been a fantastic tool to play with and using it to flag off-topic posts seems to work very well once a suitable prompt has been hammered out. Kudos to the Discourse team and @sam especially!!
The only bit about it that didn’t match my expectations is that when AI Triage/Discourse Automation is set to hide posts, the hiding seems to be done in a way such that readers sitting in the thread where the problem post resides do not see the post being hidden until/unless they refresh the topic (or navigate elsewhere and come back).
If I’ve got some users squatting in a thread and determined to have an off-topic conversation, their posts will accumulate flags (assuming the automation matches), but post hiding seems ineffective as they can still quote and reply to ostensibly hidden posts until they refresh and the posts are actually hidden for them.
I don’t know if a fix here is possible given how Discourse Automation works, but it’d be nice if its ability to hide posts was immediately effective. (I know one potential workaround is to get more aggressive with Watched Words, which I can do, but Watched Words is a pretty blunt instrument. I like the flexibility of AI triaging 
)