master ← OsamaSayegh:customize-robots-txt
merged 05:47PM - 15 Jul 19 UTC
This allows admins to customize/override the content of the robots.txt
file at …/admin/customize/robots. That page is not linked to anywhere in
the UI -- admins have to manually type the URL to access that page.
Meta topic: https://meta.discourse.org/t/needing-to-edit-robots-txt-file-where-is-it/93879?u=osama
Screenshots:
@eviltrout does it make sense to prepend a comment to robots.txt that says something along the lines of "this robots.txt file has been customized at /admin/customize/robots" **if** the file is customized? It might help with figuring out why certain things are in the file and how to remove/change them?
