Discourse Meta
Excluding user profiles in robots.txt (or allow edit of file)
Merkmal
codinghorror
(Jeff Atwood)
24. Mai 2014 um 04:19
5
User profiles are now disallowed from indexing by default.
2 „Gefällt mir“
Generic rules in "robots.txt" not picked up by Googlebot
Beitrag im Thema anzeigen
Verwandte Themen
Thema
Antworten
Aufrufe
Aktivität
How to edit robot.txt for indexing user profiles?
Support
9
901
8. Juni 2023
Pages listed in the robots.txt are crawled and indexed by Google
Support
19
3270
30. Juli 2019
Indexing User Profiles
Support
2
56
4. September 2025
Why there are lots of Disallow rule in robots.txt?
Support
34
4546
22. Dezember 2020
Generic rules in "robots.txt" not picked up by Googlebot
Support
6
997
2. April 2022