There is a setting in the ACP that says Show tags in alphabetical order. Default is to show in order of popularity but changing it has no effect on the /tags page - they are always shown by popularity by default (we want them to default to alphabetically).
Is this a bug or is this a setting for something else 
That setting was meant for the tag filter drop-down, but you’re right that it should apply to the /tags page too. I’ll add that.
Site Settings > filter by tag
Oh nice!
It would be more useful if it has a count of topics like the tags page tho - do you know if that has been suggested anywhere?
Not to my knowledge.
I pushed a fix so tags index is sort alphabetically by default if that settings is checked.