So I have imagined a new feature that could be helpful for a lot of us :
Is it possible to automate an hourly backup, only if something has changed on the precious content (new upload, new message, option changed, etc, etc) ?
To me, hourly backup is the only thing missing to make the jump : I could easily take the risk to loose one lifetime hour in exchange of very cheap hosting. I just need a strong and fast restore process, and discourse/docker install is very fast.
That could be a big bump in discourse.org popularity.
What do you think of this proposal ? Perhaps hard to implement, I don’t know.
Technically doable in a plugin but sounds VERY odd. What you would really like is some sort of delta, say every 10 minutes. That is much more technically tricky but ultimately way more useful.
As a user, I need something as simple as what is actually implemented and works really great : if something is wrong (hoster fails), I go to aws.amazon.com, download the latest backup, launch a new VM instance, install discourse via docker in a snap and restore the backup.
Probably, but I need a simple restore process.
In fact, I analyzed the risks (very low) and the cost :
Backup process is probably CPU intensive for the VM, but if the system is as smart as “do backup every hour only if something changed”, that will be a good balance to me. “Every hour” could be adapted to each case, for example : “every 3 hours” for a larger instance
S3 or alternatives like Hubic are very cheap, and their use will be optimized by the “only if” option
Two separate archives (one for the uploaded files, one other for the rest) would definitely optimize the process, but restore should be more tricky.
To be simple and because my instance is very small (60 Mo), I think the only thing I need to start on a cheaper VPS, is to be able to lower the nightly archive frequency to every one or two hours instead of every 24 hours.
Do you know if it is possible to add this hack in the app.yml ?
I don’t get why #3 exists in this example. If #1 replicates to #2 (PostgreSQL and files), wouldn’t it be easier to just copy the docker image to #2 and let #2’s PostgreSQL become the replication master when #1 fails?
@frederic: Configure PostgreSQL to store WAL files and provide disk space and limits to cover 5×N days of activity. Have a cronjob run every few minutes that fetches new WAL files and uploads via rsync. Have another cronjob run every N days that pulls a full database dump and destroys all prior WAL files and dumps (or moves them to offline backup).
You can store the necessary scripts for this in /var/discourse/shared/standalone (which gets mounted as /shared inside the container) and add lines to /etc/crontab in app.yml's custom commands section.
Well, yeah, you can use cron to run a Ruby script that loads the Discourse app and triggers the Sidekiq job, but that doesn’t solve your original requirement of only creating backups after changes.
So now, can someone help me to turn this question into ruby ?
Please sweet discourse instance, can you tell me if something has changed since one hour ?
Is there any Discourse API that I can request to know if something in the content has changed during a lapse of time ? INSERT/UPDATE/DELETE of a post or a file should be enough.
After testing the restore process from a fresh instance, Backup file not downloaded with .tar.gz extension
I would prefer a simple and strong solution, based on discourse framework.
: is it possible to trigger every one or two hours the backup sidekiq job from crontab ?
Hourra !