login_required even redirects robots.txt to /login

[michaeld (Michael - Communiteq)]

Seems like login_required is a bit too strict:

On a forum with login_required enabled, we can’t even retrieve robots.txt. That seems undesirable to me.

[codinghorror (Jeff Atwood)]

Yeah robots.txt should be immune to login required checks, I guess. Can you add that to your list @zogstrip

[zogstrip (Régis Hanol)]

Here’s the fix

https://github.com/discourse/discourse/commit/d75cc67d86c3f5042a939bc19a75e6258fc87657