Can't login after removing all bot user agents from site setting

Support
1

Disclaimer : I have 24 hours of experience with Discourse.

I just set up a new Discourse (amazing deployment experience by the way, https support was magic), in invite-only mode. I wanted to disallow crawlers and referencement and might have gone too far in this. I set allow index in robots txt to false but also removed all the bot user agents from the list in a setting that was just under. Sadly, I cannot remember the name of this setting or find it in the api. Since this, it’s impossible to access the admin panel and trying another browser doesn’t work since the login doesn’t work either (there’s no button).

the robots.txt looks fine:

User-agent: *
Disallow: /

I am not 100% confident that my emptying of that user-agent list is the root cause, so I’m open to other interpretations.
I do not have an API key yet, so I suppose I’ll have to fix this by SSH. Any idea on the steps involved to rebuild that list and fix this?

2

Have you tried logging in using https://your.discourse/u/admin-login?

3

This does indeed work for login.
It appears I do not have access to the admin and settings pages though, here’s a screenshot from the front page (no menu in headers).

1
1.png1920×1080 38.5 KB

And another from /admin.
2
2.png1920×1080 26 KB

4

Here’s how to fix your issue

# ssh into your server
cd /var/discourse
./launcher enter app
rails c
SiteSetting.crawler_user_agents = "*"

BTW, if your site is invite only, no bot will be able to crawl it :wink:

7 Likes
5

I’m unable to ssh into the machine until tomorrow, at which point I’ll let you know the result of those commands.
Thanks a lot for the quick responses.

6

This did it. Thanks a lot for the quick and concise help!

2 Likes