If it just came up in testing, I wouldn’t worry about it too much until you get more complaints from your users. My suspicion is you won’t get many if at all. Nobody complains about the page on my site.. the page itself is a power feature if you ask me and I assume nobody finds their way there unless they know what they are looking for.
I also think that organizing the tags into groups and hiding admin tags really helps - take a look at my site which also has a large number of tags but it’s fairly tidy. (keep in mind that guests and regular members don’t see all the staff tags you might be using. my tags page has dozens more that are hidden from non-staff)