# 仅阻止来自嵌入式主题的爬虫吗？

**URL:** <https://meta.discourse.org/t/block-crawlers-from-embedded-topics-only/268916>\
**Category:** Support\
**Created:** [2023年六月19日 15:34 UTC](https://meta.discourse.org/t/block-crawlers-from-embedded-topics-only/268916 "2023-06-19T15:34:53Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![angus](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/angus/32/341715_2.png) [@angus](https://meta.discourse.org/u/angus)\
**Post date:** [2023年六月19日 17:51 UTC](https://meta.discourse.org/t/block-crawlers-from-embedded-topics-only/268916/6 "2023-06-19T17:51:26Z")

</div>

嘿 @haydenjames，

在 [WP Discourse](https://github.com/discourse/wp-discourse) 插件中，您无法在 Discourse 中创建的帖子添加到 robots.txt 文件中。这实际上是一个纯粹的 Discourse 问题，即“我可以自动不对嵌入式主题使用 noindex 吗？”（或类似的话）。从 WordPress 嵌入的主题在功能上与其他嵌入式主题相同。您想探索的途径是存在的，例如 `embed set canonical url` 站点设置的来源以及相关的讨论。

> [@Add option to set canonical\_url to embed\_url](https://meta.discourse.org/t/add-option-to-set-canonical-url-to-embed-url/103915):
>
> From time to time we get requests to set the canonical URL of embedded topics to the URL of the blog post. I’ve created a [pull request](https://github.com/discourse/discourse/pull/6741) that does exactly that. It unconditionally uses the URL of the original blog post (embed\_url) as canonical for the topic. There have been various previous discussions like [Google indexed link not pointing to the correct post](https://meta.discourse.org/t/google-indexed-link-not-pointing-to-the-correct-post/61443) and [Duplicate Content](https://meta.discourse.org/t/duplicate-content/91745) in the past. After reading those posts I’m not so sure about my solution anymore, so I’d like to get some feedback f…

我认为（但欢迎纠正）您想做的事情目前不是 Discourse 的功能。Discourse 目前会向隐藏主题的 GET 请求添加 `X-Robots-Tag: noindex` 标头。您可以通过插件为嵌入式主题执行相同的操作。

---

_[View the full topic](https://meta.discourse.org/t/block-crawlers-from-embedded-topics-only/268916)._