Excluding user profiles in robots.txt (or allow edit of file)

codinghorror (Jeff Atwood) 24 مايو 2014، 4:19ص 5

User profiles are now disallowed from indexing by default.

إعجابَين (2)

Generic rules in "robots.txt" not picked up by Googlebot

الموضوع		الردود	مرات العرض
How to edit robot.txt for indexing user profiles? Support	9	886	8 يونيو 2023
Pages listed in the robots.txt are crawled and indexed by Google Support	19	3247	30 يوليو 2019
Why there are lots of Disallow rule in robots.txt? Support	34	4526	22 ديسمبر 2020
Generic rules in "robots.txt" not picked up by Googlebot Support	6	990	2 أبريل 2022
Google indexing issue (robots.txt) Support	9	675	23 مايو 2024