We’re seeing a large number of topic-permalink and pagination URLs land in Google Search Console’s “Crawled - currently not indexed” bucket (~35k URLs on a ~20-year-old imported forum, discourse version v2026.9.0-latest, self-hosted).
What we found:
Discourse serves ?page=N pagination links to crawlers for long topics (since the real UI is infinite scroll). When a JS-capable client (including Googlebot’s renderer) loads one of these ?page=N URLs, Discourse issues a client-side redirect to /t/slug//<post_number>.
Reproduction:
- Fetch a topic URL cold, with any post-number or ?page= suffix:
GET /t///14 →
GET /t// → same canonical
GET /t///1 → same canonical
GET /t//?page=127 → (self-referencing, NOT normalized) - Load the site normally (JS on) and navigate, via Ember’s router, to a URL that includes a post-number suffix (e.g. clicking a “jump to first unread post” link from Suggested Topics). Once the SPA transition completes, document.querySelector(‘link[rel=canonical]’).href equals the current post-number URL — it does not point back to the base topic.
This contradicts Discourse’s own stated design intent. In this 2015 thread, @sam was explicit:
▎ Topics: canonical should point to base topnt individual post pages from appearing asdiscrete search results.
▎ Category/list pages: canonical removed entirely, since paginated listings are not duplicates per Google’s own
▎ guidance.
What we’re seeing today breaks that in two ways:
- A topic with ?page=N (crawler pagination)ginated URL in the raw server HTML, insteadof the base topic — even though topics (unlike category lists) were explicitly supposed to always collapse to the base URL.
- After the client-side redirect from ?page=N to /t/slug/id/<post_number>, the rendered DOM’s canonical also stays on that post-number URL instead of resetting to the base topic.
Both look like regressions against the 2015 design goal, not intentional current behavior.
Why this matters for indexing: Google’s indeendered DOM for the canonical signal. Anypost-number/page-suffixed URL Google reaches directly — via the “share link to post” feature (heavily used on our forum via WhatsApp/Discord), via old external backlinks (we’re mapping legacy vBulletin ?p= permalinks to
post-number URLs), or via the crawler’s own declaring its own canonical instead ofconsolidating into the base topic. In Search Console this shows up as thousands of near-duplicate URLs stuck in “crawled - not indexed.”
Sample affected URLs from our GSC report (all last-crawled early September 2026):
/t/tengo-que-pasar-la-itv/21100/10
/t/new-honda-nsx-proyect/4910/69
/t/hacer-tubo-recto-sin-silencioso-del-catalizador-hacia-atras/25579/25
/t/kedada-japo-en-cataluna-2014/28148/75
/t/el-diario-de-patricio-de-la-plana/13286?p
Question for the team: given this is a >10 ysomething changed since that makesself-referencing canonical on these URLs intentional now? Or is this worth re-examining as a regression? Happy to provide HAR files / more repro examples if useful.