Посмотрите второй предпоследний ответ в этой теме: Needing to edit robots.txt file - where is it? - #42 by Osama.
jomaxro
(Joshua Rosenfeld)
2
Связанные темы
| Тема | Ответов | Просм. | Активность | |
|---|---|---|---|---|
| How to disable indexing by crawlers | 15 | 5294 | 18.01.2017 | |
| Pages listed in the robots.txt are crawled and indexed by Google | 18 | 3388 | 30.07.2019 | |
| Excluding user profiles in robots.txt (or allow edit of file) | 4 | 2546 | 24.05.2014 | |
| Need to find a solution to block indexing topics from a category | 2 | 99 | 27.11.2024 | |
| Malformed robots.txt causing issues with indexing | 7 | 1899 | 14.09.2019 |