Is there a way to modify the SEO settings for certain topics? For example, sometimes forum content creates what is known as “thin content” (not alot of content on a page). Google penalizes sites with alot of thin content, and it is recommended for to set a “noindex” for these types of pages. Is there a way to make certain topics as “noindex”? Also, is there a way to modify the description meta tag on a topic by topic basis? This seems to be generated automatically, and is not always done well to promote a better CTR in search (google improves rankings based on CTR relative to search volume). Thanks.
Not for topics, but if they are categories, you can make those categories require TL0 and Google can’t see them.
1 Like
You can make these topics Unlisted, this will remove them from search engines. However it also makes them (almost) invisible to users, as you need to find an explicit link from elsewhere to get to that topic now.
Honestly if the topics are short and useless… just delete them. Problem solved 
2 Likes
We create alot of topics with video instructions for customers on our forum. So we want users to be able to easily find these videos. If we make them unlisted, customers can’t find the videos as they won’t show up in the categories or tags. At the same time, however, video pages without much content, are considered thin content by Google (especially when we host these videos from YouTube), and we want to “noindex” these topics. So am I correct that there is no way presently to keep topics on the site, but at the same time “noindex” them from Google? If so, would be great to just have a simple feature like this. I come from the Wordpress world, and the way they do this is simply to have an SEO feature on a post, where you can noindex that post, but that doesn’t mean it can’t be found anywhere on the site. It’s just an SEO meta tag setting.
1 Like
You could create or commission a plugin to do that, or you could make those videos in categories available only to those who are logged in.
If you’re concerned about not having enough content, you could transcribe the videos and add them below the video embed in the topic.
5 Likes
Actually, we have been looking at the code to see how to modify some of the Meta tags, via a plugin, as they are not really ideal for SEO. Can you please let me know if I’m correct in assuming that we should be using a plugin to modify the template ‘app/views/layouts/application.html.erb’?
robots.txt is now editable for admins so you can plug the urls there.
Just visit:
https://YOURSITENAME/admin/customize/robots
So plug in the URLS there if you wish.
1 Like
Which version of discourse is this in? We don’t have this. We are on discourse 2.3.2 and it’s not available. That URL throws a 404.
This is a feature that was added after the last stable release. It’ll be available when 2.4 stable releases or if you switch to tests-passed.
3 Likes
Unfortunately beginning from 1st of September (2019) Google does not support “noindex” in robots.txt any more:
https://webmasters.googleblog.com/2019/07/a-note-on-unsupported-rules-in-robotstxt.html
So we need another way to be able to signal to Google that certain posts should not show up in the Google Search Results.
Will “unlisted” Posts show up in internal search?
Probably not.
=> Maybe there should a (core-)feature be added that any post marked as “noindex” will include noindex-meta-tags? (but still show up in internal search and be listed normally)
Any other idea?
Thanks,
Sven
3 Likes
We started to look into this, and if I’m not mistaken there are no api methods or existing widgets that can be used to modify the meta tags. So I think a plugin would be needed to override how the existing meta tags are generated. However, another solution, I was thinking about, was to just use some javascript to inject a noindex meta tag into the header, and if so, then it would be trivial to just create a theme component with a widget to do this and allow the injection on a topic by topic basis. I’m just not sure, if using javascript to do this, is a good idea, as I’m not sure how Google treats dynamic meta tag generation. Would they index the whole page and only then see the nonindex tag, making it ? I did some quick reading on this topic, and I’m honestly not sure, if using javascript to inject the tag is the right way to go. Would be interested in what more experienced developers think, as honestly I’m a newbie at Discourse.
4 Likes