So are your uploads still on the DO block storage?
Yes, all the uploads are intact.
So in that case you should be able to restore the SQL file, and then re-mount the block storage volume to get your uploads back.
There are two kinds of backups:
sql.gz which does not include uploads, and
tar.gz which does include uploads. So you had the wrong kind of backup but the fact that you had the uploads on an external volume saved your butt.
So I enter the app and restore that sql.gz, but get a permission denied error. Any idea why that might be?
You’re telling me!!
(Assuming you mean chmod). If the files are set to the wrong owner then they’re not writable.
I think this might have caused the permission denied.
What is the exact error?
Yes, thank you, I’ve been up all night and am a bit braindead.
EXCEPTION: lib/discourse.rb:93:in `exec': Failed to copy archive to tmp directory. cp: cannot open '/var/www/discourse/public/backups/default/wacky-writers-forum-2021-04-06-033906-v20210328233843.sql.gz' for reading: Permission denied
chmod 644 /var/www/discourse/public/backups/default/*
Okay I’m working my way through this now, I’ll report back shortly. Thank you for taking time to help me out.
This worked to get the restore going, THANK YOU.
Now I just have to figure out why the site still isn’t loading.
Rebuild currently in progress with saved app.yml file from before everything broke.
Is there a command to move this backup straight into the app? Restore isn’t finding it and I can’t remember how I got it to load before.
YOu can download it from S3 and put it in
You should be able to restore from the command line.
But after that, you should configure your S3 config as described in the link above; it makes things easier.
Thanks Jay. And yes that is absolutely my plan.
Okay so here’s where I am now.
- The restore was successful from that .sql.gz file. (hooray! Thanks again Richard.)
- I ensured app.yml was the same setup as before everything died
- ./launcher rebuild app
- Rebuild is successful with Postgres 13 (finally)
However, going to the site itself now is still down. I use Cloudflare but I have Development Mode on right now, and flushed the DNS cache. Everything is pointed where it’s supposed to go. The Cloudflare template is in app.yml.
DNS is resolving correctly, hostnames is up to date, the Discourse install was done with the appropriate URL, and I’m running out of ideas.
https://forum.wackywriters.com is the URL, I’m just getting “server unavailable” errors. I feel like I’m going around in circles here (sorry) but any suggestions?
Edit: When I run ./discourse-doctor, I see that there are two instances of the app running in Docker:
Is this normal? (seems like it wouldn’t be, but everything I thought I knew about Discourse has been thrown out the window the last 24 hours )
Edit2: I’ve been putting this off as a last resort, but I’m going to try and set up an entirely new server with a clean Discourse install. I’m worried something has gotten fubared with all my mucking around and I can’t figure out what’s broken. Thankfully I still have the backup and all the uploads on block storage, so if I’m lucky, I should be able to connect that to a new droplet and move things over from there. If anyone has additional suggestions or tips, I’d still appreciate more tenured expertise than mine.
Edit3: Even with a new server and IP propagating (nslookup and ping both look good, whatsmydns.net looks good), forum won’t load. Still getting connection errors. It’s like it isn’t connecting the IP address to the Discourse instance and instead is trying to load a static page, which of course, doesn’t exist in this case.
So after almost 24 hours of fighting, I figured out why the site refused to load after I got the restore going.
Because of so many resets and reinstallations and god knows what else, I hit the rate limit, so I’ve temporarily commented out the ssl templates and will get them going again in a week.
The site is “functioning” while I rebake all posts to fix the broken images but I really appreciate Jay and Richard for helping me out today, you got me through the parts I really just couldn’t figure out.
Now to get a real backup downloaded so I can get S3 setup this week without worrying about this again.
If you search, there is a way to add a second domain so that it’ll count as a separate request for let’s encrypt. But waiting is easier.
I recommend that you put cloudflare to gray cloud with no speedups.
@pfaffman Aren’t you confusing object storage with block storage? Object storage is s3, but TS says they used block storage, which is just a disk mounted at their uploads directory:
Yeah. So nothing I said makes any sense.
Thanks for noticing that, Richard.
Well, most things you said did make sense but you had me confused here
Well, they made perfect sense, they just didn’t apply to the problem at hand. Thanks for the catch!
I’m okay with waiting, everything seems to be working okay for now.
I will take your advice and do so, thanks again for all your help.
It’s okay, I knew what you meant. I’m planning to move us to Digital Ocean Spaces so technically I have to add “S3” to app.yml.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.