Посмотрите второй предпоследний ответ в этой теме: Needing to edit robots.txt file - where is it? - #42 by Osama.
jomaxro
(Joshua Rosenfeld)
2
Связанные темы
| Тема | Ответов | Просм. | Активность | |
|---|---|---|---|---|
| How to disable indexing by crawlers | 17 | 5258 | 24.04.2019 | |
| Pages listed in the robots.txt are crawled and indexed by Google | 19 | 3344 | 30.07.2019 | |
| Excluding user profiles in robots.txt (or allow edit of file) | 5 | 2537 | 24.05.2014 | |
| Need to find a solution to block indexing topics from a category | 2 | 92 | 27.11.2024 | |
| Malformed robots.txt causing issues with indexing | 9 | 1883 | 21.04.2023 |