Invite users to certain categories/topics of a closed forum

There are no permissions at the topic level. You could make a topic unlisted in a public category.