As you noticed, creating a new public topic that links back to a private message doesn’t make much sense, and it won’t embed. However, the cross-topic linkage WILL be visible for those that can view both topics.
The button is removed because it won’t embed.
In terms of fundamental differences, a PM is still a real topic and contains real posts. The only thing it doesn’t have is a category.