No content on homepage for Googlebot

Bug
1

Currently there doesn’t appear to be any content output in the #main-outlet element for Googlebot:

Pasted image1113×718 51.1 KB

HTML for reference:


<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Discourse Meta</title>
    <meta name="description" content="Discussion about the next-generation open source Discourse forum software">
    <meta name="author" content="">
<meta name="generator" content="Discourse 1.5.0.beta12 - https://github.com/discourse/discourse version e84d5549efe6c24a6230cfbe2ce8c1b511ffc9d6">
<link rel="icon" type="image/png" href="https://assets-meta-cdck-prod-meta.s3.dualstack.us-west-1.amazonaws.com/original/3X/c/b/cb4bec8901221d4a646e45e1fa03db3a65e17f59.png">
<link rel="apple-touch-icon" type="image/png" href="https://assets-meta-cdck-prod-meta.s3.dualstack.us-west-1.amazonaws.com/original/3X/c/b/cb4bec8901221d4a646e45e1fa03db3a65e17f59.png">
<link rel="icon" sizes="144x144" href="https://assets-meta-cdck-prod-meta.s3.dualstack.us-west-1.amazonaws.com/original/3X/c/b/cb4bec8901221d4a646e45e1fa03db3a65e17f59.png">
<meta name="theme-color" content="#ffffff">
<meta name="viewport" content="width=device-width, minimum-scale=1.0, maximum-scale=1.0, user-scalable=yes">


      
    
        <link rel="alternate" type="application/rss+xml" title="Latest posts" href="https://meta.discourse.org/posts.rss" />
    <link rel="alternate" type="application/rss+xml" title="Latest topics" href="https://meta.discourse.org/latest.rss" />

    <style>
        img { max-width: 100%; width: auto; height: auto; }
    </style>
  </head>
  <body>
      
    <header>
      <a href="/"><img src="https://assets-meta-cdck-prod-meta.s3.dualstack.us-west-1.amazonaws.com/original/3X/9/d/9d543e92b15b06924249654667a81441a55867eb.png" alt="Discourse Meta" id="site-logo" style="max-width: 400px;"></a>
    </header>
    <div id="main-outlet" class="wrap">
      



    </div>
    <footer class="container">
      <nav itemscope itemtype='http://schema.org/SiteNavigationElement'>
        <a href='/'>Home</a>
        <a href="/categories">Categories</a>
        <a href="/guidelines">FAQ/Guidelines</a>
        <a href="/tos">Terms of Service</a>
        <a href="/privacy">Privacy Policy</a>
      </nav>
      <p>Powered by <a href="http://www.discourse.org">Discourse</a>, best viewed with JavaScript enabled</p>
    </footer>
    <script type="text/javascript">

  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-33736483-2']);


  _gaq.push(['_setCustomVar', 1, 'Anonymous', true, 2]);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();

</script>

       <script>
 Discourse.ExternalNavItem = Discourse.NavItem.extend({
    href : function() {
      return this.get('href');
    }.property('href')
  });

  I18n.translations[I18n.locale].js.filters.bugs = { title: "Bugs", help: "Open Bugs" };
  I18n.translations[I18n.locale].js.filters.features = { title: "Features", help: "Open Feature Requests" };
     
  Discourse.NavItem.reopenClass({
    buildList : function(category, args) {
      var list = this._super(category, args);
      if(!category) {
        list.push(Discourse.ExternalNavItem.create({href: '/category/bug?order=op_likes&status=open', name: 'bugs'}));
        list.push(Discourse.ExternalNavItem.create({href: '/category/feature?order=op_likes&status=open', name: 'features'}));
      }
      return list;
    }
  });
  </script>
  </body>
</html>

I would expect the latest list of topics for meta.

1 Like
2

How did you set the user agent of Googlebot? My Chrome doesn’t have that choice and I wonder if it’s related.

3

Chrome Extenstion-

I am using this. Getting okay results as of now.

4

As per this post:

I added a “custom device” to Google Chrome dev tools using the newly proposed Googlebot smartphone user agent for mobile:

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

1 Like
5

Huh interesting, I can reproduce it while in incognito. Looking into it.

1 Like
6

I found this because I was running this new Mobile Friendly Google Tool against my Discourse instance:
https://www.google.com/webmasters/tools/mobile-friendly/

I was expecting the output to be clunky - but expected topic results.

1 Like
7

Looks like it’s a combination of mobile + Googlebot. We had an optimization to never render the <noscript> content in mobile mode. I’ve changed it so that we only skip rendering that if we aren’t in our crawler layout. Seems to work!

https://github.com/discourse/discourse/commit/233aa2ab2f2942f2e1c33338ea4f0868f32b6f53

5 Likes