Using official ad plugin Official Advertising / Ad Plugin for Discourse The /new - /unread pages having crawler isssue, how to solve this? [image]

Can you share your url? Does your site require login?

Actually, its our clients website, it does require login, but the website is accessible to all without login same as this forum meta.discourse.org

So if you’re not logged in you can see the topics at mysterysite.com/new?

No, it is not visible! Oops! That page doesn’t exist or is private.

So that’s what Google sees. I’m pretty sure that it’s frowned on to have content listed that’s not available, though the paywall sites do that to some degree. Edit: I got this one wrong!

In order to determine what content is “new” or “unread”, the forum needs to know who you are, and thus requires a logged in user.

Adsense will work fine without access to these two pages.

Shouldn’t these two be excluded in robots.txt?

Adsense ad crawler errors

Support

rlauriston (Robert Lauriston) July 17, 2023, 5:00pm 11

AdSense works but I get no revenue from the ads that would appear on that page. What values should I specify for Login URL and Login method?

Topic		Replies	Views
How to Display Google Adsense ads on login-protected pages Support login , advertising	3	121	March 16, 2025
Can search engines crawl, if forum behind single sign on Support	4	843	January 11, 2020
AdSense plugin comparison Support	6	2155	July 15, 2019
Understanding anonymous metrics on a login required site Data & reporting	6	1189	February 21, 2022
Finding an alternative to adsense Community Building	20	2329	March 10, 2024

Adsense ad crawler errors

Related topics