What do you exactly mean by that? Moderators have to be well-trusted members, in order to review both public and private flags. When someone flags a message, they simply ask for a moderators’ or administrators’ help. Keep in mind, that moderators can only see private messages when flagged.
To answer the title question: How to setup moderators with just access to remove or update topic categories you can select a group of members to be category moderators. In this way, you limit their admin actions. I am not sure though wheter they will have the ability to remove topics from their assigned category. You can give them some other admin abilities though by promoting them to Trust Level 4.