Category moderators do have a selection of elevated powers within their designated categories that aren’t available to any trust level, though (rightly or wrongly) I think there is an assumption that they would also be trusted members of the community and would work in conjunction with the higher trust level features (You can check out a comparison table in Trust Level Permissions Table (inc Moderator Roles)).
I think allowing category moderators to set slow mode independently of being a TL4 would make sense though, and would make a good feature request.
