# robots.txt に記載されたページは Google によってクロールおよびインデックスされます

**URL:** https://meta.discourse.org/t/pages-listed-in-the-robots-txt-are-crawled-and-indexed-by-google/100309
**Category:** Support
**Created:** [2018 年 10 月 23 日午後 7:23 UTC](https://meta.discourse.org/t/pages-listed-in-the-robots-txt-are-crawled-and-indexed-by-google/100309 "2018-10-23T19:23:49Z")
**Posts on this page:** 1
**Showing post:** 11

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2018 年 10 月 24 日午前 10:28 UTC](https://meta.discourse.org/t/pages-listed-in-the-robots-txt-are-crawled-and-indexed-by-google/100309/11 "2018-10-24T10:28:39Z")

</div>

Huh, it looks like `noindex` may be supported in `robots.txt` already?

> **[Robots.txt Noindex Update: Everything SEOs Need to Know](https://www.lumar.io/blog/best-practice/robots-txt-noindex-the-best-kept-secret-in-seo/)**
>
> Following the recent update to the robots.txt protocol, find out everything you need to know about using the noindex directive in your robots.txt.

And it seems [it works](https://www.stonetemple.com/does-google-respect-robots-txt-noindex-and-should-you-use-it/)

> Ultimately, the NoIndex directive in Robots.txt is pretty effective. It worked in 11 out of 12 cases we tested. It might work for your site, and because of how it’s implemented it gives you a path to prevent crawling of a page AND also have it removed from the index.

cc @sam this will be the easiest way.

---

_[View the full topic](https://meta.discourse.org/t/pages-listed-in-the-robots-txt-are-crawled-and-indexed-by-google/100309)._