That could be potentially bad idea — unless you are totally sure every other bots can be disallowed. Google is using a lot crawlers that aren’t declaring googlebot string.
Jagster
(Jakke Flemming)
5
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Google Not Indexing 95k posts | 4 | 198 | April 21, 2025 | |
| 1 million Topics - Takes millions of days to get indexed without Sitemap in Robots | 3 | 2722 | February 20, 2017 | |
| Google Search Indexing and Discourse | 9 | 4018 | June 9, 2020 | |
| Why isn't Google Indexing Discourse? SEO concerns | 31 | 6385 | June 1, 2024 | |
| Google search indexing not working? | 1 | 151 | January 4, 2026 |