I already removed them by hand! or let’s say, I could remove one of the duplicates for each pair; and then the page for the other tag in the pair became in-accessible (even for admins)! as we go to the page, we get “that page doesn’t exist or is private”, though there are several topics with that tag.
an example would be tag یادگیری which should have 488 topics in it, while the tag-page doesn’t exist (link to the tag page):
p.s.: sorry the pages and the tags are non-English. I hope it gives a clue.
I also think this is a #bug, and the bad part is that it’s ongoing! no matter I have deleted previous duplicates, I get new ones again. if it helps I can remove the non-official plugins and check the birth of new duplicates.
if it was due to Farsi language, I’d suggest it may be because of some of apparently similar but basically different alphabets that different keyboards are using. for example some devices use Arabic alphabet and some use Persian. and in these keyboards we have ي and ی, which when used in a word one can’t distinguish among them (but they are different in encoding).
but @Pfaffman 's case was for English indexes, bot Persian!