I think this is really similar in spirit to this old topic: Show recipient in "sent" personal messages.
I don’t think most users will look at it like “I want to see the set of topics that this set of users that includes me share private access to”. In practice we just want to be able to quickly find some moderation (or other type) of message that the other guy didn’t reply to, and they can be potentially hard to find at the moment.
The workaround we’ve been using has been just bugging people to make trivial responses they otherwise wouldn’t make.