How to allow user-agent access to private discourse?

Support
4

AFAIK there should be no need to as long as your Security Setting is
allow_index_in_robots_txt: true

I’m wondering if it isn’t so much a crawl thing and that they need to be allowed in the robots.txt file or if it is a widget permissions thing.