Group page indexing


(Evgeny) #1

Is it possible to index a group page?
https://meta.discourse.org/groups

In the file

https://meta.discourse.org/robots.txt
Summary
User-agent: *
Disallow: /auth/cas
Disallow: /auth/facebook/callback
Disallow: /auth/twitter/callback
Disallow: /auth/google/callback
Disallow: /auth/yahoo/callback
Disallow: /auth/github/callback
Disallow: /auth/cas/callback
Disallow: /assets/browser-update*.js
Disallow: /users/
Disallow: /u/
Disallow: /badges/
Disallow: /search
Disallow: /search/
Disallow: /tags
Disallow: /tags/
Disallow: /email/
Disallow: /session
Disallow: /session/
Disallow: /admin
Disallow: /admin/
Disallow: /user-api-key
Disallow: /user-api-key/
Disallow: /*?api_key*
Disallow: /*?*api_key*

The page is not closed for indexing

As the search engine sees it

 </header>
 <div id="main-outlet" class="wrap">
        <!-- preload-content: -->
        empty
 
        <!-- :preload-content -->
        <footer>

Is that a mistake or is that supposed to be?

2


(Sam Saffron) #2

This is a minor #bug but we have to decide first if we even want “groups” page to be google indexed?


(Jeff Atwood) #3

I would say no, go ahead and add groups to default robots exclusion


(Sam Saffron) #4

Done here:


(Jeff Atwood) #5