Yes/No actions require the user to parse the full question and how it maps onto the actions provided. This significantly slows things down, especially if similar questions are asked in rapid succession – which is exactly what happens in the mod queue:
First, “Yes” means not spam, then “Yes” means yes spam.
To fix this, the approach usually is to replace the yes/no buttons with concrete actions. In processes, this most commonly is OK/Cancel, in the moderation context, it probably should be Approve/Delete.