Would be nice if this could be used to upload to a different S3 compatible provider, such as running MinIO.
If you want to use minio, see Using Object Storage for Uploads (S3 & Clones). If you want different services for backups and assets then you’re out of luck (though there are ways to set up triggers to copy from one bucket 8 another)
No, just want for backups, so they get automatically pushed to a different machine on my network.
Then the guide I linked is what you’re looking for.
Is it possible to be more frequent than once per day? I’d rather not make assumptions and set backup freqeuncy to a float
If you want more frequent backups you’ll need to do them externally.
Thanks for the hint. That pushed me towards the command line option which we can schedule to do whenever:
I got this to work, but it seems that the uploads checkbox was not really needed, nor do I understand the purpose. What is the purpose? The only thing I want is backups to s3 instead of local for my server. The server only has weekly automatic backups…
The Json also had problems… I was able to get it to work using another website reference. However, nobody could upload any images because I had the uploads checkbox checked (as described here)…Unchecking that box fixed the image upload problem for users and their profile pics.
What is the purpose of the images upload? I’m seriously hoping images are inside the s3 backups.
I had to do the instructions twice because I didn’t understand “uploads” and only made one bucket. Then I had to do it again with 2 buckets, and then I had to remove the checkbox for uploads. It might be good if there was a separate more simple topic for s3 backups … and only backups.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:List*",
"s3:Get*",
"s3:AbortMultipartUpload",
"s3:DeleteObject",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:PutObjectVersionAcl",
"s3:PutLifecycleConfiguration",
"s3:CreateBucket",
"s3:PutBucketCORS"
],
"Resource": [
"arn:aws:s3:::classicaltheravadabucket",
"arn:aws:s3:::classicaltheravadabucket/*",
"arn:aws:s3:::classicaltheravadabackupbucket",
"arn:aws:s3:::classicaltheravadabackupbucket/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets",
"s3:*"
],
"Resource": "*"
}
]
}
Though I think that topic should be updated to recommend that the S3 configuration be moved to app.yml
rather than the database, that way you can do a command line restore of the database with only the yml file and not have to configure it with a user and s3 confige before doing a restore.
I’m not sure what you are talking about . My backups are working see picture
.
I use s3 because digital ocean backups are only weekly, and if the server crashes and gets deleted, it is not useful.
On the other hand. I’m hoping that restoring from s3 or downloaded s3 bucket will be fine.
I’m not uploading the images, and I am hoping that the s3 backups are being backed up including images (although very few).
Generally: no.
It does not make much sense to backup images in an S3 bucket into another S3 bucket?
can you be less ambiguous?
The instructions had 2 s3 buckets. I could not get that to work.
I have only one s3 bucket. Hopefully pictures are included in that backup… is that correct?
I would imagine that local backups work the same, Right?
Please answer in full sentences regarding my questions. The tutorial was very confusing as well.
What is ambiguous about “no” ? (And what’s not ambiguous about “backups being backed up” )
Let me try again.
If you have configured uploads to be on S3 then uploads are not included in your backup.
Let’s use the term “pictures” instead of uploads even though it can be other media.
This way we don’t confuse the text content as an upload which I’m uploading to s3.
So the 62 MB, backup files on s3 as pictured and uploaded in this thread does not include pictures ?
So how do I make sure the backups have these?
Do the local backups have the pictures too?
When I configured the s3 for "uploads (of media) " which was ambiguous as well. Nobody could post pictures because they were rejected from s3…
Is there a way to have both local and s3 daily backups?
I could care less if 5 days of pictures were lost, we are mostly a text based group.
But I would care if 5 days of text were lost. Digital Ocean only does 7-day backups if you pay them.
So even though I can backup daily, if the droplet gets hacked or damaged, then we lose those backups… I’m starting to think there is not much added value in s3.
I wish there were simple backups similar to WordPress which lets me backup to my google or dropbox account.
No, that is a bad idea, if you upload a text file as an attachment, it’s an upload as well, it will cause confusion. And text in a post is stored into the database. So I’m sticking to the term uploads.
If your uploads are on S3, they are not included in backups. In that case the backups only contain a copy of the database. It does not matter whether your backups are local or on S3.
If your uploads are not on S3, they are included in the backups. In that case the backups contain a copy of the database, and a copy of the uploads. It does not matter whether your backups are local or on S3.
If you are storing something on S3, be it uploads or backups of the database, they will not get lost if your DO droplet gets hacked or damaged. So I don’t see your point.
Since your posts are about backups and not about file and image uploads, I’m moving these to another topic.
I’d like to automatically move my S3 Backups to Glacier but I’m confused by the steps linked in the first post, which doesn’t explain much, maybe because there’s outdated stuff.
Which options should be checked here?
May I ask again in case someone did these steps and knows about it?
Also, do you know what makes these fluctuations in S3 fees?
Plus, since the launch of the forum (September 2020), the size of the backups has increased by roughly 15%, but the S3 bills have doubled, from 2.50$ to 5$. Any idea why that much?
That’s why I’d like to use Glacier.
Edit: I’ve followed the steps described here and I’ll see how it goes.
Well, it doesn’t go.
My lifecycle configuration:
My S3 bucket:
No backup is on Glacier.
So… Two questions for those who have been able to achieve this automated S3 to Glacier transition:
-
What could be wrong in my configuration?
-
The minimum storage duration charge in Glacier is 90 days. Does that mean that if I do 1 backup a day, I’ll be eventually charged for 90 backups in Glacier each month?
If this is the case, then this Glacier solution won’t be a good idea, unless I reduce a lot my backups frequency.
where in the vps are the backups stored?
I added this to the OP: