Excluding user profiles in robots.txt (or allow edit of file)

codinghorror (Jeff Atwood) 2014 年 5 月 24 日午前 4:19 5

User profiles are now disallowed from indexing by default.

「いいね！」 2

Generic rules in "robots.txt" not picked up by Googlebot

トピック		返信	表示
How to edit robot.txt for indexing user profiles? Support	9	901	2023 年 6 月 8 日
Pages listed in the robots.txt are crawled and indexed by Google Support	19	3270	2019 年 7 月 30 日
Indexing User Profiles Support	2	56	2025 年 9 月 4 日
Why there are lots of Disallow rule in robots.txt? Support	34	4546	2020 年 12 月 22 日
Generic rules in "robots.txt" not picked up by Googlebot Support	6	997	2022 年 4 月 2 日