No, there is currently not a way to alter the permissions, as Jeff said in this topic:
Moderators for a specific category is on the roadmap, but not yet implemented.
This is what moderators can do vs, admins: