Is this really a bug?
When you type ' or ", we auto-convert it to ‘ or “, to match how it will be cooked. If you really want to keep the originally typed character, you backspace once.
This is the same rule for most other input rules – when text input is converted to something else, it can usually be undone with the backspace.
We can make these smart quotes not undoable, in this case the backspace would just remove it, but if you type " again, it will be converted again, making it impossible to type the unconverted version – probably not an issue.
Another option is… just dropping smart quote replacement altogether in the rich editor, meaning we wouldn’t have the correct preview of them before posting.
@lindsey thoughts?