This morning around 5:35am, my forums suddenly spiked on disk usage and crashed, going completely offline. I had to resize the Digital Ocean image to get them back on their feet. Oof. Here’s my disk usage over the last 24 hours: [image] Question: what kind of logs/post-mortem analysis can I look…

Yeah, backups are a common source of this kind of failure - what’s the disk usage look like over a 7 day window? Also note that local uploads are included in these backups, so if you had a significant increase in uploads around 18:00, that would also increase the backup archive size.

Hmm. I have been transitioning files off of S3 and back to my local server , but the process to do that seems to run on the fly, and only a few hundred images (all around ~300k each) at a time = ~0.1 GB per batch. Over the last week, I might have run the script 20 times, so 20 batches = around 2 GB t…

Hmmm. My backups are set to go to S3, every 2 days, but nothing since the 9th? [image] Discourse ‘Backups’ view [image] Amazon S3 view [image] Incidentally, after seeing the above, I clicked the button in Discourse to trigger a backup. It took 28 minutes and seemed to work fine; I now see tha…

[이미지] bartv: I’d start with figuring out which directory is blowing up. My standard approach is to enter /var/discourse and then run du -h -d 1 . Take the largest directory, enter it and repeat until you find the suspect. Once you have it, that might give you a clue to what’s going on. I’m my…

You have spikes to almost 100% every night before — looks like this one tipped you over the edge. I expect the previous backups failed due to out-of-space while creating the local backup file to send to S3, but merely failed, and didn’t break your forum. You finally noticed when the out-of-space mad…

Aha, so those little spikes are Discourse trying to make a backup! That sheds some light on things. So here is my trailing 7-day chart again. [image] Maybe what we’re seeing is: Several times over this past week, Discourse attempted to make a backup. This process temporary eats a lot of disk …

It didn’t necessarily get further along last night; you just “lost a race” — what happens when you run out of space depends on what component first is affected by the problem. If it fails to make a backup, I rather expect it might try to send a message, but if it’s out of disk space, it might not s…

[이미지] mcdanlj: I don’t know how you are judging “inordinate” — it doesn’t strike me as inordinate. Fair enough; as you point out, there are many variables in play. [이미지] mcdanlj: A fixed percentage doesn’t really tell you much; the database might be tiny compared to uploads, or vice vers…

Estimating space required for backup is one of the hard problems in computer science… It’s a distant relative of the progress bar. :wink: In all seriousness, part of it is a database dump, and I don’t know how you would estimate that ahead of time. If you have enough images that space becomes an is…

백업 중 디스크 사용량 급증, Discourse가 완전히 충돌했습니다 :(

제품 지원 자체 호스팅

pnoeric (Eric) 7월 23, 2020, 12:51오후 22

브라이언 (그리고 @Stephen) 감사합니다. 여러분이 조언해 주신 대로 새 버킷을 설정해 두었습니다. 이제 올바른 설정을 찾고 있는 중입니다.

글		댓글	조회수
Configure automatic backups for Discourse Self-Hosting backups , how-to	44	304976	5월 19, 2026
I can't get a good backup - it fails after after_create_hook Bug	16	2987	4월 25, 2016
Set up BackBlaze S3 with BunnyCDN Sysadmins how-to	34	7316	4월 17, 2025
How to backup Discourse when The backup has failed? Self-hosting	8	1114	8월 5, 2020
Backups are duplicating and not respecting number to keep on disk Self-hosting	67	3190	2월 14, 2019

백업 중 디스크 사용량 급증, Discourse가 완전히 충돌했습니다 :(

Related topics