Even though you can’t set the setting per category, since you can restrict certain tags to certain categories, you won’t have ALL tags showing in certain categories. It’s not exactly what you probably want, but close, I believe.
I don’t think, for my personal workflow, that limiting the max of tags per category would make much sense, because as I mentioned, I like to know what I have available. I don’t mind scrolling. And if I know I want a specific tag, I can just type it.
I also don’t think that having 500 tags will do any good to the community, for example, so even 30 tags or so is probably enough? I don’t know… I’m just starting my 2 communities, so let’s see how that goes.