Other than manually, is it possible to set up robots.txt to block all WP-Discouse-connected forum threads? (so that the forum post isn’t indexed) A simple toggle option that can block all connected forum threads from crawling? Or even a per-post toggle setting when publishing or editing a WP post …

[תמונה] Hayden James: Other than manually, is it possible to set up robots.txt to block all WP-Discouse-connected forum threads? (so that the forum post isn’t indexed) Why do you want to do that? I’m fairly certain that the forum post includes a pointer to the canonical version on your WP si…

A drop in ranking/reputation of WP’s articles (originals) since addition. I was not looking to start a debate or complain, just asking how to achieve this. I’m sure there are many other reasons others may want to have this level of control for what’s indexed.

[תמונה] Hayden James: I was not looking to start a debate or complain, just asking how to achieve this Just suggesting - perhaps you should re-define your goals because you are heading to wrong direction. But with reverse proxy that is quite trivial task.

Not all web crawlers honor the robots.txt file.

Hey @haydenjames , There’s nothing you can do in the WP Discourse plugin to add posts it creates in Discourse to a robots.txt file. This is actually just a pure Discourse question, namely “Can I automatically noindex embedded topics?” (or something along those lines). A topic embedded from Wordpress…

Heading in the wrong direction to block indexing of a forum thread with the duplicate article that I prefer Google search users to find via the WP blog? I’m ok with that. The Wp-discourse main benefit for me has been allowing discussions of blog posts without having to use solutions like Disqus or t…

[תמונה] Hayden James: with the duplicate article There isn’t any duplicates unless you have changed something. Because of: [תמונה] Jay Pfaffman: to the canonical version on your WP site And: [תמונה] Angus McLeod: A topic embedded from Wordpress is functionally the same as any ot…

[תמונה] Angus McLeod: I don’t think (but happy to be corrected) that what you want to do is a current Discourse feature. Discourse currently adds a X-Robots-Tag: noindex header to GET requests for hidden topics. You could do the same for embedded topics via a plugin. Thanks @angus To clarify…

[תמונה] Jakke Flemming: There isn’t any duplicates unless you have changed something. …depends on your definition of duplicate. Canonical is in place, but for me personally, since both the Blog post and the forum thread contain the exact paste/text (duplicate). I would like just to block thos…

A “hidden” topic is one that isn’t listed on topic lists, i.e. it isn’t “discoverable” in the normal fashion. You can tell a hidden topic by the eye with a line through it symbol. [Screenshot 2023-06-19 at 20.37.37] [תמונה] Hayden James: To clarify, if I make the category that stores the WP-d…

לחסום זוחלים רק מנושאים משובצים?

תמיכה

angus (Angus McLeod) 19 ביוני,‏ 2023,‏ 6:58pm 14

@haydenjames The one final thing I’d note is that there seems to have been an issue with the canonical url of embedded topics recently. Something to keep in mind if you just noticed this issue recently.

נושא		תגובות	צפיות
Old topics published via WordPress embed are missing x-robots: noindex and canonical tags WordPress wordpress	9	232	5 ביוני,‏ 2026
Need a way to add "meta robots noindex" in topics from a category Feature seo	6	1298	12 בפברואר,‏ 2025
Posts in WP Discourse interferes with the SEO of the Wordpress site WordPress	2	639	24 בנובמבר,‏ 2021
Google changed how they process robots.txt in Discourse? Support	20	1795	22 בדצמבר,‏ 2020
Discourse SEO overview (sitemap / robots.txt) Site Management seo , explanation	0	2291	18 באוקטובר,‏ 2023

לחסום זוחלים רק מנושאים משובצים?

נושאים קשורים