# Googlebot 404 errors due to page numbers
**URL:** https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337
**Category:** Support
**Created:** [September 4, 2018, 3:46pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337 "2018-09-04T15:46:41Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [September 4, 2018, 3:46pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/1 "2018-09-04T15:46:41Z")
I’ve got a site that’s gotten the “Googlebot identified a significant increase in the number of URLs” error.
It looks like the bad urls (returning 404s) are all of the form `https://site/t/slug/id?page=XXX`.
And changing `XXX` to `XXX-y` makes the URLs work fine. (I thought it was an off-by-one error at first, but sometimes `y` needs to be significantly more than 1).
It’s not immediately apparent that it’s due to deleted posts, which was my next guess.
Perhaps if the page number is greater than the number of pages there should be a 301 to page 1 or something? (I don’t pretend to know anything about SEO). Or should I tell them just not to worry about those 404s?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [September 4, 2018, 10:21pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/2 "2018-09-04T22:21:27Z")
This is only an issue we can address if you can find actual cases on the forum where the meta tags link to non existent pages
If the content is looking good and google are just having an adventure adjusting to the fact that a book once had 100 pages and now has 10 there is not much we can do, a 301 or 302 here is not ideal, perhaps we can add it for deletions… it’s a giant edge case though
---
### Author:  [@stas00](https://meta.discourse.org/u/stas00)
#### Post date: [February 9, 2019, 10:27pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/3 "2019-02-09T22:27:59Z")
Found this topic after encountering the same issue.
I fail to see how this is google’s problem, you can’t expect every page to be reindexed every hours.
Are you seriously saying that discourse can’t check that if a link includes `?page=2` which would translate to /21, but if the topic shrunk to 19 posts, it can’t handle that and redirect to the last post /19 in this case?
In fact, discourse already has everything ready to do it right, w/o needing to put the blame on external forces. Currently, If the topic has 19 posts and I type /99, it automatically redirects to /19 - so it knows how to handle shrinking topics just fine. So if you let ?page=2 to always redirect to /21, w/o checking whether there are 21+ posts or not, discourse will do the right thing after 2 redirects:
1. …?page=2 =\> Redirect …/21
2. …/21 =\> ../19
of course, I’m assuming the server is configured to handle multiple redirects.
Does this make sense?
p.s. I think the proposed solution reduces the edge case to general case and actually simplifies things, since you no longer need to check whether ?page=2 “exists” and let redirects do the right thing.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [February 9, 2019, 11:07pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/4 "2019-02-09T23:07:43Z")
Make sure you are on latest (2.2 final or 2.3 beta) as an important fix went out for web crawler pagination about a month ago.
---
### Author:  [@stas00](https://meta.discourse.org/u/stas00)
#### Post date: [February 9, 2019, 11:37pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/5 "2019-02-09T23:37:24Z")
We are already using 2.3.0.beta2, so that did not fix the issue.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [February 10, 2019, 1:51am UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/6 "2019-02-10T01:51:19Z")
Can you provide live URLs demonstrating the problem?
---
### Author:  [@stas00](https://meta.discourse.org/u/stas00)
#### Post date: [February 10, 2019, 2:08am UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/7 "2019-02-10T02:08:49Z")
Of course, e.g. take the first hit from [this search](https://www.google.ca/search?num=100&source=hp&ei=38tdXOraHtWT0PEPuP6v2A0&q=PeakMemMetric&btnK=Google+Search&oq=PeakMemMetric&gs_l=psy-ab.3...893.893..1708...0.0..0.137.266.0j2......0....2j1..gws-wiz.....0.SppVhDDYwTQ), which links to [this](https://forums.fast.ai/t/gpu-optimizations-central/33944?page=2). (assuming google will show you the same results, but if not, then just use the last link of my reply directly.)
if you remove page=2 or use page=1 it works.
Thank you for looking into it, @codinghorror!
---
### Author:  [@stas00](https://meta.discourse.org/u/stas00)
#### Post date: [February 10, 2019, 5:48am UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/8 "2019-02-10T05:48:28Z")
Actually, you don’t need live examples from elsewhere - take this thread on your own server, it doesn’t do too well either: [Googlebot 404 errors due to page numbers](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337?page=2) - same problem.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [February 10, 2019, 8:48pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/9 "2019-02-10T20:48:41Z")
That topic was split so you destabilized the page number, it is a fundamental limitation with our numbering implementation
Take a book analogy, in a book if you rip out page 77, page 200 remains page 200. In Discourse page 200 becomes page 199
I guess, maybe it does make sense to auto redirect high page numbers back to last page.
---
### Author:  [@stas00](https://meta.discourse.org/u/stas00)
#### Post date: [February 10, 2019, 9:45pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/10 "2019-02-10T21:45:39Z")
> [@sam](#):
>
> That topic was split so you destabilized the page number, it is a fundamental limitation with our numbering implementation
This very [topic](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/) wasn’t split and has [the same problem](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337?page=2). Therefore the way how the shrinking came about matters not.
> Take a book analogy, in a book if you rip out page 77, page 200 remains page 200. In Discourse page 200 becomes page 199
but discourse is not a book which is cast in “paper” once printed, it’s dynamic “living” system. As such I am unable to see where the book metaphor fits in this case.
And even if I run with your metaphor, page 200 remains page 200, so if there was a link to page 200 before that worked - it should work still if pages 1-199 were ripped out. Except, this surely is not the way to fix this problem, which is much simpler if you don’t look at it as a book.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [February 10, 2019, 10:03pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/11 "2019-02-10T22:03:30Z")
> [@stas00](#):
>
> This very [topic](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/) wasn’t split and has [the same problem](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337?page=2). Therefore the way how the shrinking came about matters not.
Hmmm 🤔 not seeing any `page=2` here at all, when I Google:
`"Googlebot 404 errors due to page numbers" site:meta.discourse.org`
> [@stas00](#):
>
> it should work still if pages 1-199 were ripped out.
Hence I said
> I guess, maybe it does make sense to auto redirect high page numbers back to last page.
The risk here is that a change like this masks bugs.
---
### Author:  [@stas00](https://meta.discourse.org/u/stas00)
#### Post date: [February 10, 2019, 10:10pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/12 "2019-02-10T22:10:30Z")
> [@sam](#):
>
> > This very [topic](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/) wasn’t split and has [the same problem](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337?page=2). Therefore the way how the shrinking came about matters not.
>
> Hmmm 🤔 not seeing any `page=2` here at all, when I Google:
>
> `"Googlebot 404 errors due to page numbers" site:meta.discourse.org`
Apologies for not being clear, @sam. I meant that if this topic were to shrink from 21 to 20 posts it would have the same issue, and thus you have your own setup to test with and no need for external live URLs.
And I demonstrated how this fails by [this direct link](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337?page=2).
I gave this example, because @codinghorror initially suggested that the server might not be running the latest code base.
> Hence I said
>
> ```
> I guess, maybe it does make sense to auto redirect high page numbers back to last page.
>
> ```
>
> The risk here is that a change like this masks bugs.
+1
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [February 27, 2022, 2:05pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/13 "2022-02-27T14:05:03Z")
---
### Author:  [@rahim123](https://meta.discourse.org/u/rahim123)
#### Post date: [October 18, 2023, 6:49am UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/14 "2023-10-18T06:49:51Z")
This is still a problem.
> [@Sitemap plugin - no ?page=… urls in default sitemap](https://meta.discourse.org/t/sitemap-plugin-no-page-urls-in-default-sitemap/220283/10):
>
> I’m having trouble with Google Search Console trying to index URLs like https://example.com/t/title-slug/1234?page=3 , which make Discourse throw a 404. Removing the ?page=x parameter makes the URL valid. I assume this is some kind of a side effect of Discourse adding pagination to the version of the site that it serves crawlers:
> [@sam](#):
>
> I guess, maybe it does make sense to auto redirect high page numbers back to last page.
This would seem to be a decent solution. But I don’t understand why this is happening in the first place? Why is Discourse adding page numbers that don’t exist to the sitemap?
> [@sam](#):
>
> That topic was split so you destabilized the page number, it is a fundamental limitation with our numbering implementation
>
> Take a book analogy, in a book if you rip out page 77, page 200 remains page 200. In Discourse page 200 becomes page 199
In the case of the topics I’m examining with this issue there are no deleted posts or other modifications to the threading.
---
### Author:  [@Pothi](https://meta.discourse.org/u/Pothi)
#### Post date: [May 15, 2026, 10:00am UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/15 "2026-05-15T10:00:20Z")
> [@sam](#):
>
> I guess, maybe it does make sense to auto redirect high page numbers back to last page.
Yes, it’d be a nice to have such as a solution. When there is a 404 for a particular page number, it could be redirected to pageNumber-1 .
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [May 21, 2026, 2:48pm UTC](https://meta.discourse.org/t/googlebot-404-errors-due-to-page-numbers/96337/16 "2026-05-21T14:48:16Z")
Hopefully this will fix the issue without introducing new ones 😅
[https://github.com/discourse/discourse/pull/40224](https://github.com/discourse/discourse/pull/40224)