The same issue has popped up on my site.
In addition to that, it occurred to me that the editor in a private message exchange looks exactly the same:
Solutions could include adding an extra button to the post menu, so that you can choose whether to reply publicly or privately. An icon like this could work.
https://www.flaticon.com/free-icon/email_480334?term=reply&page=1&position=87&page=1&position=87&related_id=480334&origin=search
Another idea would be to add a “reply privately” button at the bottom of the editor.