The discussion revolves around adding semantic microdata and more content for crawlers. fantasticfears suggests adding more microdata to topic views, lists, category lists, and FAQ pages to improve search engine display. This includes adding category descriptions and links to category lists. fantasticfears also requests adding the 360Spider crawler to the UA list, a top Chinese search engine, which codinghorror agrees to.
fantasticfears shares a GitHub commit with changes, which codinghorror reviews, suggesting not to link the “Top” page as it’s duplicate from a crawler’s perspective. fantasticfears proposes adding <meta name="robots" content="nofollow"> for the top page to allow accessibility for noscript crawlers like Lynx. However, codinghorror doubts the necessity of these crawler-only pages.
Breadcrumb is one of many semantic markup described in http://schema.org. Search engines rely on these markups to improve the display of search result.
This feature request is to add more things for crawler.
More microdata can be added into topic view, lists, category list and faq page. Maybe it can help for displaying topic list in the search engines.
Beyond that, add some content for crawler in some views.
Add category description in the category list.
Add link to category list in topic list page. There is no link to get there.
BTW, can I add 360Spider to crawler ua list? It’s the second search engine in China now. (appr. Baidu 56% - 360 30%. bye to the third search engine for the last 10 )
)