Restrict replies to a certain group of users

Permissions are at the category level. You want them at the topic level. Changing that will be very difficult.