How do I disable outgoing email partially?

Are you not using reply by email?