When browsing a discourse site (meta.discourse.org) without JavaScript in the same way as it is presented to Googlebot several links are missing.
Links to user profiles
Link to categories list from homepage
Links to users activity from users page
Links to related topics at the bottom of a topic.
Any actual activity on users activity/topics page: Profile - sam - Discourse Meta (yes this looks the same as the users page because it is, no different content.)
EDIT: The idea being if a user would find it hard to navigate to / see related interesting info - so would a search engine.
Irrelevant. Check robots.txt, user profiles are not allowed to be indexed anyway.
This also addresses your points #1, #3, and #5@DeanMarkTaylor.
Suggested topics is indeed random once you’ve iterated through all the stuff that requires a user login, such as “is this new?” and “am I tracking this?”. So that rules #4 out as well.
I don’t think any of the links mentioned need to appear in the JS-off page.
There are two important considerations when using /robots.txt:
robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.
So don’t try to use /robots.txt to hide information.
So though Google may not ignore the suggestion, other bots will, and robots.txt alone will not be enough to deter Profile SPAMners as indeed it hasn’t yet.