Backup and Restoration Planning

Product Support

AI-generated summary

The discussion revolves around backup and restoration planning for a Discourse site. LotusJeff outlines their current backup strategy, including daily database backups, file backups, and server backups, and asks for feedback. Jagster suggests increasing database backup frequency to every 6 hours, citing risk reduction, and shares their cron setup using docker exec app discourse backup --sql-only to create additional backups to S3. pfaffman confirms this method and suggests PostgreSQL replication for critical data. LotusJeff clarifies whether the cron method replaces UI backups, and Jagster confirms they use both methods, prioritizing redundancy over backup volume. The conversation highlights the importance of frequent database backups and the use of cron jobs to enhance data protection.

1

I want to ensure a good backup strategy as my site goes live. These are the following steps I have put in place. What am I missing?

Database Backups

  • Discourse takes one daily database backup.
  • An additional cron database backup is performed daily. (+12 hours from UI-based backup)
  • Backups are stored in an S3 bucket (different data center)

Core Files

The following are backed up twice a day to an S3 bucket (different data center)

  • discourse/containers/app.yml
  • discourse/shared/standalone/uploads/
  • discourse/shared/standalone/backups/
  • discourse/shared/standalone/ssl/
  • discourse/shared/standalone/letsencrypt/

File backups occur twice a day, 30 minutes after database backups.

S3:Uploads

  • Daily backup of Uploads is stored in an S3 bucket in a different data center.

Server Backups

  • Weekly backups of the entire server - 4 weekly backups kept
  • Annually store one weekly backup as a master backup in a remote location.

This should provide all the necessary data and settings to restore a server in place or on a new server.

Am I missing anything?

2

That is suggested way. If you backup your database once a day you are risking max 24 hours worth of what ever happened in that forum.

I’m told at least twice it isn’t any issue, but no one has never explained why not. So I’m backing up my database every 6th hour — my forum isn’t so busy, so I can take that risk. To compare — my e-commerce does backup every 4th minute.

3

How did you set up your database to back up more frequently? I would prefer twice a day, but the UI functionality only allows daily.

4

You can have a cron job (on your server, not in the container) run

docker exec app bash -c "discourse backup"

If the data are really critical, then you can set up a postgres replication server and have every commit on a second server.

5

I have this is crontab:

docker exec app discourse backup --sql-only

That is Discourse’s own CLI command of backuping, and I was guided that docker exec app executes it outside of container (app on name of the container, of course.

And because I’ve configured S3 that jumps into same bucket where ”normal” backups are too.

There is one small problem… quite soon there is zillion backups. I don’t know if I should do differently sql-dump, move it using aws-cli and then killing everything that is older than some timeframe. Or do same thing in VPS.

But that is the most easiest way to get sql-dump.

6

I am assuming this activates the internal discourse backup routine, so all notifications remain in place.

Do you turn off the backup scheduling within the UI and manage all via cron? or, are you doing one within the UI and doing the additional via cron?

7

No. I do both. I don’t trust system, and myself, that much. Quite rarely amount of backups is real issue, shortage is.

8

Thank you @Jagster and @pfaffman for the assist in setting an additional database via cron. That moves my system down to a worst case 12 hour data loss.