This isn’t true, at least not in my experience- whispered posts aren’t visible, even as a “hidden reply”, to users without access. I just tested it to make sure; a topic with one whispered reply shows no replies (it doesn’t even show the infobox that gets added with replies) to a non-moderator