RGJ
(Richard - Communiteq)
9월 1, 2026, 3:36오후
1
고객 중 한 명이 관리자 UI를 통해 로컬 Discourse 백업을 업로드할 때 재현 가능한 문제에 직면했습니다.
백업 업로더는 고정된 5MB 청크를 사용하며, 충분히 빠른 연결 환경에서는 Discourse의 자체 애플리케이션 레벨 요청 속도 제한을 초과하는 충분한 수의 요청을 생성할 수 있습니다. 이 경우 서버는 HTTP 429를 반환하지만, 청크 업로더는 429 응답을 재시도하지 않으며 Retry-After 헤더도 무시합니다. 그 결과 전체 백업 업로드가 실패합니다.
비 2xx 응답은 오류 로 변환됩니다:
if (ev.target.status < 200 || ev.target.status >= 300) {
const error = new Error("Non 2xx");
error.source = ev.target;
reject(error);
return;
}
재시도 로직은 다음과 같습니다:
_shouldRetry(err) {
if (err.source && typeof err.source.status === "number") {
const { status } = err.source;
return (
status === 0 ||
status === 409 ||
status === 423 ||
(status >= 500 && status < 600)
);
}
return false;
}
따라서 HTTP 429는 명시적으로 재시도 가능한 상태가 아닙니다.
429 응답 하나만으로도 청크 업로드 프로미스가 실패하게 되며, 결국 전체 백업 업로드에 대해 upload-error가 트리거됩니다.
또한 Retry-After 응답 헤더에 대한 처리도 없습니다.
2021년에 작성되었으나 빠르게 방향이 어긋나버린 이 토픽을 찾았는데, 정확히 동일한 문제인 것 같습니다.
@RGJ 보고해 주셔서 감사합니다 다음 PR로 수정될 예정입니다
feature/rate-limit-client-helpers ← fix/retry-rate-limited-backup-chunks
merged 10:52AM - 15 Sep 26 UTC
Stacked on #43372. Fixes the report in https://meta.discourse.org/t/chunked-back… up-uploader-exceeds-discourse-request-rate-limit/411386.
Uploading a local backup through the admin UI fails outright on a fast connection. The uploader sends one POST per chunk, so throughput alone decides the request rate, and a large backup over a quick link trips the per-user request limiter. The server answers 429 and the upload dies, discarding however many gigabytes had already been sent. There is no resume, so a retry starts from chunk 1.
`_shouldRetry` allowed status 0, 409, 423 and 5xx. That list came from uppy's S3 multipart plugin, where it is correct — S3 does not answer 429 with a `Retry-After`. Discourse does, so the one status worth retrying was the one treated as fatal, and none of the four configured retry delays was ever consumed.
### Why three changes and not one
Adding 429 to the allowlist alone makes it worse. The limiters run in aggressive mode, so a rejected request re-arms the window; retrying on the existing sub-second ladder turns a fast failure into a permanent block. Measured against a persistent 429 on a 6-chunk file:
| | POSTs of the failing chunk |
|---|---|
| allowlist only | 15 in 10s, still fails |
| \+ honour `Retry-After` | 6 |
| \+ keep the chunk reserved | 2 — correct |
The third is needed because clearing `busy` before rejecting let every sibling completion re-pick the same chunk and start a second retry chain that POSTed immediately, defeating the backoff.
### Chunk size stays at 5 MB
An earlier revision of this PR raised it with the ladder S3 multipart has used since #22061. @Ethsim12 caught that this regresses stock installs, and reproduced it: these chunks go through the site's own nginx, whose shipped `client_max_body_size` is `10m`, and the multipart envelope puts a 10 MB chunk 962 bytes over that. nginx answers 413 before Rails sees the request, and 413 is not retryable — so every backup of 100 MB or more would have died on chunk 1. S3 parts have no proxy in front of them, which is why the ladder is correct there and stays there.
### Latent defects fixed alongside
- Registering the abort listener called `xhr.abort()` immediately and registered its return value, so aborting was a no-op and in-flight chunks kept running after a failure was reported.
- `chunkSize` was only assigned when no `getChunkSize` was supplied, so any caller providing one produced NaN bounds, a single empty chunk, and a silently truncated backup.
- The pause plumbing called a method the class never defined; nothing could reach it, so it is gone.
### Not a regression from the Uppy migration
Worth recording, since the linked topic asks. The 2021 report predates Uppy becoming the default backup uploader by two days — that reporter was on Resumable.js, which classified 429 as retryable but retried immediately, up to 100 times, against a limiter that was already aggressive. 429 has never been survivable here. What the port changed was the failure mode, from slow-and-noisy to instant-and-silent. The capability actually lost was resume, which this PR does not restore.
3개의 좋아요