Thanks for being so clear that I am wrong. 
I’m not suggesting that users will want to have all 100+ categories in their sidebars. But users often engage across a wide variety of categories and the names of the sub-categories are relative to the parent category and don’t make as much sense out of context.
So using site ordering for each user’s chosen categories will be the difference between usable and unusable for my site’s users.
Yes, we’re also working on reducing the number of categories; but even a sane-for-site-purpose number of categories would be many dozens, so the core problem remains.