RGJ
1. September 2026 um 15:36
1
Einer unserer Kunden ist auf ein reproduzierbares Problem gestoßen, als er ein lokales Discourse-Backup über die Admin-Oberfläche hochgeladen hat.
Der Backup-Uploader verwendet feste 5-MB-Blöcke (Chunks) und kann bei einer sufficiently schnellen Verbindung genügend Anfragen generieren, um das anwendungseigene Request-Rate-Limit von Discourse zu überschreiten. In diesem Fall gibt der Server HTTP 429 zurück, aber der Chunked-Uploader versucht 429-Antworten nicht erneut und beachtet den Retry-After-Header nicht. Infolgedessen schlägt der gesamte Backup-Upload fehl.
Nicht-2xx-Antworten werden in einen Fehler umgewandelt:
if (ev.target.status < 200 || ev.target.status >= 300) {
const error = new Error("Non 2xx");
error.source = ev.target;
reject(error);
return;
}
Die Retry -Logik lautet:
_shouldRetry(err) {
if (err.source && typeof err.source.status === "number") {
const { status } = err.source;
return (
status === 0 ||
status === 409 ||
status === 423 ||
(status >= 500 && status < 600)
);
}
return false;
}
HTTP 429 ist daher ausdrücklich nicht wiederholbar.
Eine einzige 429-Antwort reicht aus, damit das Chunk-Upload-Promise fehlschlägt, was schließlich upload-error für den gesamten Backup-Upload auslöst.
Auch wird der Retry-After-Response-Header nicht verarbeitet.
Ich habe dieses Thema aus dem Jahr 2021 gefunden, das ziemlich schnell in andere Bahnen gelenkt wurde, aber anscheinend exakt dasselbe Problem beschreibt.
daemon:
Ich habe gerade Discourse auf einem neuen Server installiert und versuche jetzt, ein Backup von etwa 400 MB hochzuladen.Slow down, too many requests from this IP address. Please retry again in 2 seconds. Error code: ip_10_secs_limit.
zogstrip
8. September 2026 um 08:03
3
Danke für den Bericht @RGJ
feature/rate-limit-client-helpers ← fix/retry-rate-limited-backup-chunks
merged 10:52AM - 15 Sep 26 UTC
Stacked on #43372. Fixes the report in https://meta.discourse.org/t/chunked-back… up-uploader-exceeds-discourse-request-rate-limit/411386.
Uploading a local backup through the admin UI fails outright on a fast connection. The uploader sends one POST per chunk, so throughput alone decides the request rate, and a large backup over a quick link trips the per-user request limiter. The server answers 429 and the upload dies, discarding however many gigabytes had already been sent. There is no resume, so a retry starts from chunk 1.
`_shouldRetry` allowed status 0, 409, 423 and 5xx. That list came from uppy's S3 multipart plugin, where it is correct — S3 does not answer 429 with a `Retry-After`. Discourse does, so the one status worth retrying was the one treated as fatal, and none of the four configured retry delays was ever consumed.
### Why three changes and not one
Adding 429 to the allowlist alone makes it worse. The limiters run in aggressive mode, so a rejected request re-arms the window; retrying on the existing sub-second ladder turns a fast failure into a permanent block. Measured against a persistent 429 on a 6-chunk file:
| | POSTs of the failing chunk |
|---|---|
| allowlist only | 15 in 10s, still fails |
| \+ honour `Retry-After` | 6 |
| \+ keep the chunk reserved | 2 — correct |
The third is needed because clearing `busy` before rejecting let every sibling completion re-pick the same chunk and start a second retry chain that POSTed immediately, defeating the backoff.
### Chunk size stays at 5 MB
An earlier revision of this PR raised it with the ladder S3 multipart has used since #22061. @Ethsim12 caught that this regresses stock installs, and reproduced it: these chunks go through the site's own nginx, whose shipped `client_max_body_size` is `10m`, and the multipart envelope puts a 10 MB chunk 962 bytes over that. nginx answers 413 before Rails sees the request, and 413 is not retryable — so every backup of 100 MB or more would have died on chunk 1. S3 parts have no proxy in front of them, which is why the ladder is correct there and stays there.
### Latent defects fixed alongside
- Registering the abort listener called `xhr.abort()` immediately and registered its return value, so aborting was a no-op and in-flight chunks kept running after a failure was reported.
- `chunkSize` was only assigned when no `getChunkSize` was supplied, so any caller providing one produced NaN bounds, a single empty chunk, and a silently truncated backup.
- The pause plumbing called a method the class never defined; nothing could reach it, so it is gone.
### Not a regression from the Uppy migration
Worth recording, since the linked topic asks. The 2021 report predates Uppy becoming the default backup uploader by two days — that reporter was on Resumable.js, which classified 429 as retryable but retried immediately, up to 100 times, against a limiter that was already aggressive. 429 has never been survivable here. What the port changed was the failure mode, from slow-and-noisy to instant-and-silent. The capability actually lost was resume, which this PR does not restore.
3 „Gefällt mir“
wird behoben durch