I have migrated from the Discourse hosting to self host but I lost all the images in my forum, I realised it very late and it has been more than 30 days for this migration.
When I realised it, I ran the following code but it did not work:
`> ./launcher enter app
> rake posts:rebake
`
I talked to Discourse support but they told me that they can not restore the backup as this has been more than 30 days. What can I do to fix this issue:
I hate to say it, but there might be nothing you can do here.
If you didn’t take a full backup which included images, and you didn’t realize past the point at which your host had purged their own backups, then the data is probably lost.
If you don’t mind me asking, what happened for it to take a month before you realized your images weren’t in the backup? Was the site down for a whole month?
No, the site was active but I got busy with other work. Also, I do take full back from the Discourse Admin panel.
If your assets were being stored on remote storage such as s3, then they aren’t going to be included in the backup.
There’s also the backup with uploads setting which skips uploads.
How big is the backup file?
You probably downloaded your backup before you canceled your account and it didn’t include the uploads.
You might ask again very nicely if they can temporarily restore the uploads to S3. I had this happen a while back when they got around the cleaning out their old S3 buckets and they were able to restore the uploads to S3 for a bit.
If they can, you can then contrive to download those images from S3 and then re-upload them to your own S3 bucket, or otherwise contrive to get them into discourse. It’s something of a chore, but it is possible.
This is likely; the link to the final full backup includes the S3 uploads, but other backups do not.
I can’t find any previous support requests from this account. From an account with the email associated with your former site, please PM @team with the name of the site, and I can double check to ensure the uploads are no longer available.
You don’t notice that, since the migrated site keeps loading the images from the CDCK S3, until they get removed, and then it’s too late.
We also have a S3 bucket version lock on our uploads, so even after they’ve been deleted we can still restore them for a period of time.
I grasp how S3 works, it’s usually a huge plus when moving a site between servers. I was referring to the relatively huge difference in file sizes.
We have forums with a huge database and (relatively) few uploads. I would be completely unable to predict how large the backup of such a forum would be, nor spot the difference between a backup with and without backups, especially with only one of the two being present.
I guess I just hark from an older generation. My ops days began with stacks of tapes and the guiding principle that it isn’t a backup unless you’ve validated that it works and contains the expected data. Nothing like spending hours pulling data off LTO and realizing that permissions or file locks blocked a backup agent from grabbing critical files.
If the current arrangement can bamboozle new and seasoned users alike maybe we need a clearer indicator in the filename, appending complete and no-uploads ? The existing approach is giving users a false sense of security, so this definitely feels like there’s room for improvement.
Would you be open to that @supermathie?
Well, I think we’re from the same generation, and I wouldn’t be falling for that either.
But on one hand we (as a community) keep telling people they should go ahead and self host, “it’s easy, just follow instructions” and now we expect them not to only check the size of their backups, but also view the contents of a .tar.gz file, something they’ve probably never heard of, nor know how to do that. And even if they do, it’s hard to tell when the contents are sufficiently there. Most forums that have their uploads on S3 do have some local files as well. It’s not that black and white that either all files, or no files are there.
The problem with restoring a forum where the uploads are left on the (previous) S3 location is that it does work. Everything looks fine, until the uploads disappear a month later. And even if files are missing, doesn’t the restore process tell you that it’s normal that “you will see missing images” and everything will be fine eventually?
So let’s improve that too?
Maybe we need to be detecting when a backup is being restored to a fresh instance and warn the user that assets weren’t included in the restore they just ran.
I’m pretty sure we will improve this. A warning when you restore a backup without uploads makes a lot of sense.