Host directive in robots.txt


(Oleg Bovykin) #1

Hi! Today I got error from Yandex Webmaster about error in robots.txt. Its about missing Host directive.

As I understand, host is available through ENV[‘DISCOURSE_HOSTNAME’] variable. If I right, its really simple to add host directive to robots.txt. If there are no underwater stones, I can send pull request.