You could post a notice to all the users (a pinned banner on your homepage?) telling them not to reply to a post if they flag it due to the fact their reply will also be hidden until the flag is resolved.
That’s a good idea so long as the user that was flagged allows PM from other users. If in their preferences they’ve unchecked the box Allow other users to send me personal messages in their Preferences/Notifications/Messages, users who flag their posts cannot send them a PM to explain the reason for the flag. ![]()