The site settings reply_by_email_enabled and reply_by_email_address are not showing up in Admin - Email settings. Funny enough, the alternative_reply_by_email_addresses setting does show.
It seems they’re missing an area key in site_settings.yml although I don’t understand how they do show up in Admin - All Site Settings - Email.