Group page indexing

Is it possible to index a group page?
https://meta.discourse.org/groups

In the file

https://meta.discourse.org/robots.txt

Summary
User-agent: *
Disallow: /auth/cas
Disallow: /auth/facebook/callback
Disallow: /auth/twitter/callback
Disallow: /auth/google/callback
Disallow: /auth/yahoo/callback
Disallow: /auth/github/callback
Disallow: /auth/cas/callback
Disallow: /assets/browser-update*.js
Disallow: /users/
Disallow: /u/
Disallow: /badges/
Disallow: /search
Disallow: /search/
Disallow: /tags
Disallow: /tags/
Disallow: /email/
Disallow: /session
Disallow: /session/
Disallow: /admin
Disallow: /admin/
Disallow: /user-api-key
Disallow: /user-api-key/
Disallow: /*?api_key*
Disallow: /*?*api_key*

The page is not closed for indexing

As the search engine sees it

 </header>
 <div id="main-outlet" class="wrap">
        <!-- preload-content: -->
        empty
 
        <!-- :preload-content -->
        <footer>

Is that a mistake or is that supposed to be?

2

1 Like

This is a minor bug but we have to decide first if we even want “groups” page to be google indexed?

3 Likes

I would say no, go ahead and add groups to default robots exclusion

1 Like

Done here:

https://github.com/discourse/discourse/commit/e19ae6c55eb254809152e23c3ebd401bb05360ee

6 Likes