I don’t recognize all category colors myself, but I am familiar with the colors of the categories I frequent, which makes it a little easier to pick them out of a list.
I think “this is a subcategory” can be valuable. There are also cases where a site has multiple subcategories with the same name, so in that case the color could serve as the most obvious differentiation for a mention. It definitely adds some noise, and I’m sure people have different preferences for signal:noise ratio.