Doing it as suggested should only give them moderator-like abilities within that specific category. 
I think Trust Level Permissions Reference still gives a nice overview of what extra abilities are granted by being a category moderator (vs ones granted from trust levels, etc).