Perhaps one way to look at this predicament is as a kind of meta-moderation. We allow users to flag posts, such that moderators can review the flags. Inevitably the mods will see how many flags, and may have some sense of who the flaggers are. Users who overuse flags or misuse flags need to be detected so that their behaviour can be brought to moderator attention.
Perhaps some sort of reputation could be maintained, hidden of course, which relates to
- how many flags a user issues
- how many distinct accounts make posts they are flagging
- whether their flags are accepted or rejected
- whether their own posts are flagged
A user with poor flagging reputation needs some kind of attention, some of which could perhaps be automated.