Configuring reply_by_email breaks regular email sending

The first step in investigation is to understand the problem. What does “broke” mean, exactly?