Hi Andy! Man, sorry this is such a struggle. The link to your site Public Happiness tags page helps a bit, methinks - you have alot of weird issues there as you can see… no tag groups which is good to know. If you sort by name (alphabetically) you will see there are blank tags that appear to be after D… pretty weird and points to some larger issue.
Can you confirm that you are seeing the same issue in safe mode, with only official plugins and no theme customizations enabled?
Have you rebuilt your instance having removed/commented out in app.yml the unofficial tags plugins you don’t want? Maybe they are still having some influence.
I am suspecting all the blanks are occurring each time you run into trouble saving with multiple tags, including the ones that are obviously weird because they are showing up multiple times on the tags page. You could test this by loading the tags page, trying to replicate it again, and then see if a new one is added.
Beyond the above.. if I were you, I’d start by doing some tag gardening. 
Generally you don’t want to have so many tags, and definitely no tags that will likely only appear once. You can delete and merge tags via the tags pages. I would go ahead and delete the ones that are appearing in duplicate, or merge them with a new temporary tag, make sure the originals are deleted, and then rename it back to the tag name you want.