If I remember right that limit comes a bit deeper than robots.txt. Sure, it is quite fast test allowing /u in robots.txt. But if it comes deeper from the core you may need a plugin.
Dig here. That was a thing perhaps a year ago, came along with SEO-stuff, and there was strong reason why it is not allowed.