Site setting keys are not translateable?

Of course. Just as it would be technically possible to write HTML, CSS and JavaScript in Hindi and map it to English.

I don’t see this as being that big of a problem as long as the values are translatable. And this is in the Admin pages after all.

If this is important to you why not write a “translated key mapped to umtranslated key” plugin?