I personally find the UX around whispers confusing.
To create a whisper, I “Toggle whisper [on]”.
Once I post my whisper, my next reply defaults to whisper mode. Okay, that’s kinda cool. But to turn whisper mode back off, I would expect to re-toggle it (toggles generally do go in 2 directions…). Instead I have to “Reply to topic”.
I believe clicking Toggle whisper again also turns it back off, are you trying this and it is not working? See a gif below where it works just fine for me.
Before I’ve posted the first whisper I can toggle it back and forth. It’s when I reply to a whisper that there’s a problem:
Welcome back @ganncamp 
This is by design. If you’re replying to a public post, your reply can either public or as a whisper. If you reply to a whisper, your reply will have to be a whisper.
It’s not possible to reply to a whisper publicly.
When I read a thread, I read all the way to the bottom. And then, once I’m completely caught up, I’m ready to respond either publicly or privately.
So I hit the closest “Reply” button.
If the last item in the thread happens to have been a Whisper, then I seem to be “stuck” in whisper mode. (It took me quite a while to figure out the right dropdown option to break out of that.)
Surely it’s not by design that after having read my colleagues’ back-channel chatter I must choose the correct Reply button when I’m ready to respond to the user?
In the forum I moderate, in a case where, like you, I have read all the way to the bottom I hit the reply button for the topic, not for the individual post.
That’s because I am not replying to that particular post (in your case a whisper post) but to the topic itself. In that case it works as one would expect.
I love the Whisper function, it has made it possible to collaborate with the other mods in a super fast way and everybody is looking at the same thing. Just great.
I do the same as well.
If you’re replying to the topic - either publicly or as a whisper - you should use the topic reply buttons here.
Both of those are “reply to the topic” buttons where you can toggle whispers on/off.
If the last post is a whisper and you click here
Your reply will always be a whisper because you’re replying to the post, not the topic.
Thanks for the input.
I was attempting to report a perceived UX problem. The consensus seems to be that the problem is on my end. Most of you have learned the tool. Silly me thinking that I shouldn’t have to. That it should just work.
The reason it is that way, is because people replied to a whisper and expected it to be a whisper – when it wasn’t… they were very, very embarrassed.
So right now, the rule is that a reply to a whisper is always a whisper, to prevent any possible embarrassment on behalf of the user. We view it as a safety feature. 
On our legacy forum before we migrated to Discourse, we wrote a custom plugin (over 10 years ago or so) we called “confidential posts” which is very similar to the built-in Discourse feature “whispers”.
More-often-than-not, a moderator made a “confidential post” comment to other moderators, and then forgot, inadvertently, to insure their reply was also "confidential’ causing “social issues”.
Hence, based on our experience before Discourse, as @codinghorror describes, it is an important feature to insure that replies to “whispers” are always “whispers”.
In my view, it’s not so much an issue of "learning the UI’ as much at it is “having experience with various forum management situations and scenarios (that arise from time-to-time)”; because experienced forum managers who use similar functions such a “confidential posts”, “whispers” or “underground posts”, learned years ago “what is a whisper, should stay a whisper”.
I totally get that. Appreciate it even. 
The thing is that getting back out of whisper mode is difficult to find.
I’ve written a simple theme component to convert a post to whisper or vice-versa, after its posted.
Sure, I just wanted to clarify that we weren’t doing arbitrary-rules-for-the-sake-of-rules here… someone got badly burned by this in the past and “design for safety” is one of the principles we try to build into Discourse whenever we can 
.
