The question of invitations relates to groups rather than categories. Group managers (as opposed to category moderators) have been implemented on the back-end but not exposed in the UI. But you could use that to set up specific users that can invite users into topics that are in restricted categories.