Semantic microdata and more content for crawlers

Feature
1

Breadcrumb is one of many semantic markup described in http://schema.org. Search engines rely on these markups to improve the display of search result.

This feature request is to add more things for crawler.

  • More microdata can be added into topic view, lists, category list and faq page. Maybe it can help for displaying topic list in the search engines.

Beyond that, add some content for crawler in some views.

  • Add category description in the category list.
  • Add link to category list in topic list page. There is no link to get there.

BTW, can I add 360Spider to crawler ua list? It’s the second search engine in China now. (appr. Baidu 56% - 360 30%. bye to the third search engine for the last 10 :cold_sweat: )

2 Likes
2

Sure add 360 UA that is fine.

1 Like
3

Does this looks fine to you? @codinghorror

https://github.com/fantasticfears/discourse/commit/4ede84758e5d4536b6fc50c4440bce374fa6e115

1 Like
4

I would not link Top, that is entirely duplicate links from the crawler’s perspective. The rest seems fine.

5

What about <meta name="robots" content="nofollow"> for top page? Then the top page can be accessible for noscript, eg Lynx.

6

I seriously doubt anyone is using those crawler-only pages.

1 Like