This morning around 5:35am, my forums suddenly spiked on disk usage and crashed, going completely offline. I had to resize the Digital Ocean image to get them back on their feet. Oof. Here’s my disk usage over the last 24 hours: [image] Question: what kind of logs/post-mortem analysis can I look…

Yeah, backups are a common source of this kind of failure - what’s the disk usage look like over a 7 day window? Also note that local uploads are included in these backups, so if you had a significant increase in uploads around 18:00, that would also increase the backup archive size.

Hmm. I have been transitioning files off of S3 and back to my local server , but the process to do that seems to run on the fly, and only a few hundred images (all around ~300k each) at a time = ~0.1 GB per batch. Over the last week, I might have run the script 20 times, so 20 batches = around 2 GB t…

Hmmm. My backups are set to go to S3, every 2 days, but nothing since the 9th? [image] Discourse ‘Backups’ view [image] Amazon S3 view [image] Incidentally, after seeing the above, I clicked the button in Discourse to trigger a backup. It took 28 minutes and seemed to work fine; I now see tha…

[изображение] bartv: I’d start with figuring out which directory is blowing up. My standard approach is to enter /var/discourse and then run du -h -d 1 . Take the largest directory, enter it and repeat until you find the suspect. Once you have it, that might give you a clue to what’s going on. …

You have spikes to almost 100% every night before — looks like this one tipped you over the edge. I expect the previous backups failed due to out-of-space while creating the local backup file to send to S3, but merely failed, and didn’t break your forum. You finally noticed when the out-of-space mad…

Aha, so those little spikes are Discourse trying to make a backup! That sheds some light on things. So here is my trailing 7-day chart again. [image] Maybe what we’re seeing is: Several times over this past week, Discourse attempted to make a backup. This process temporary eats a lot of disk …

It didn’t necessarily get further along last night; you just “lost a race” — what happens when you run out of space depends on what component first is affected by the problem. If it fails to make a backup, I rather expect it might try to send a message, but if it’s out of disk space, it might not s…

[изображение] mcdanlj: I don’t know how you are judging “inordinate” — it doesn’t strike me as inordinate. Fair enough; as you point out, there are many variables in play. [изображение] mcdanlj: A fixed percentage doesn’t really tell you much; the database might be tiny compared to uploa…

Estimating space required for backup is one of the hard problems in computer science… It’s a distant relative of the progress bar. :wink: In all seriousness, part of it is a database dump, and I don’t know how you would estimate that ahead of time. If you have enough images that space becomes an is…

Резкий скачок использования диска во время резервного копирования, Discourse упал :-(

Поддержка Самостоятельное хостинг

pnoeric (Eric) 23.Июль.2020 12:51:27 22

Спасибо, Брайан (и @Stephen), я настроил новый бакет точно так, как вы советовали. Теперь я только подбираю правильные настройки.

Тема		Ответов	Просм.
Configure automatic backups for Discourse Self-Hosting backups , how-to	44	303465	19.05.2026
I can't get a good backup - it fails after after_create_hook Bug	17	2885	21.09.2016
Set up BackBlaze S3 with BunnyCDN Sysadmins how-to	34	6763	17.04.2025
How to backup Discourse when The backup has failed? Self-hosting	8	1000	05.08.2020
Backups are duplicating and not respecting number to keep on disk Self-hosting	68	2630	15.02.2019

Резкий скачок использования диска во время резервного копирования, Discourse упал :-(

Связанные темы