Hi folks - in the process of trying to migrate from hosted Discourse to self-hosting. Have done a bunch of successful test runs of various things, but when I try to do the real thing including all uploads, after a couple of hours unzipping the archive it says that no such file or directory exists when attempting to extract the dump file. So now facing the prospect of losing around 140GB of uploads unless anyone has any ideas?
Can you provide the log?
Are you restoring from the command line or the web interface? I recommend the web interface.
Hi - I attached log to previous email, although attached again here. I tried it first using the web interface and then second time via command line. I have a suspicion that the backup might be corrupt in some way, as it is not recognised when uploaded to S3 and is rejected almost instantly if I try to upload it via the browser.
restore-failure-log.txt (3.28 KB)
That’s what it looks like. Did you upload with the web browser or scp/rsync? I would upload again with rsync.
Hi Jay - sorry I got a bit mixed up earlier as we have also been in discussions with Discourse via email throughout this migration process, and it was there that I attached the log file.
Looking at the error, I suspect that the tarball doesn’t actually contain a sql dump, just images. The file was initiated and checked by Discourse on our behalf. I downloaded it via http and uploaded to the server via scp, as the browser upload rejected it.
Yeah just ran a command to inspect the contents of the tarball and it only contains images, no SQL dump.
Can you verify whether the sizes of the tarball are exactly the same?
- on the CDCK instance
- the downloaded one
- the one you uploaded using scp
You might want to tar tfvz to check if the archive is not truncated.
You might want to check if you’re not running out of disk space somewhere, since it needs a multiple of the size of the archive.
Ok I have popped out for a bit but will check later. Space should be no issue as have 512GB and the backup file is 70GB. I was surprised that the file was a few GB smaller than the last one we created, I was expecting it to be slightly larger. Pretty sure that for some reason it has not included the SQL dump, which would be about the difference in size.
Hi @pfaffman @RGJ just an update on how this has progressed. The SQL dump was indeed missing from the downloaded archive, and I wasn’t sure whether taking a separate DB backup and inserting it into the archive would work (and would take several hours to test). So we just ended up restoring the database and migrating (successfully).
Problem now is that as soon as Discourse decommission everything and shut off the S3 bucket / CDN, all our historic images will break.
I have all the images, and think I can hopefully just upload them all to our S3 bucket maintaining the same folder structure. Seen a few threads discussing the possibility of using discourse.remap / dbhelper.remap to bulk update the links at database level. Any thoughts on that much appreciated!
I can’t imagine how that could have happened. Did your browser somehow uncompress and un-tar the backup and you tried to put it back together again?
You can ask the discourse.org folks to give you a backup that includes the uploads. That’s what you want to do. They turn on a include_s3_uploads_in_backup (that’s close to, but almost certainly not the name of the hidden setting).
You can also contrive to use S3 tools to pull them all down from their bucket and push them back up. There are a couple of topics about that. I don’t recommend it.
I recently migrated a 100GB-ish backup from CDCK to Digital Ocean, droplet, spaces bucket, and bunny.net CDNs for $1000. I was sorry.
Is that just the database?
Oh, did you somehow do a database-only restore even though you have the images in the tar file?
You need to do the restore of the exact file that they made and have Discourse restore it. The one that has the database and the uploads. Or you can look at the restore code and contrive to do by hand the stuff that it does to remap the images to the new location. Though I think Richard has skills and tools to do that, I don’t think you want to do it that way.
We did a test run a couple of months back and all was fine. I think they left that hidden setting on, as this time I was able to trigger a backup including uploads from the back-end, although after about 12 hours we got a notification that it had failed. I then contacted Discourse and they said they would create the backup from their end. A couple of hours later the backup I had initiated seemed to complete, although we discarded the file as advised by Discourse. They then hit a bunch of issues with backups timing out and returning errors, before eventually telling us there was a complete file. But when I tried to restore the file, after taking several hours to extract the archive it complained that the dump was missing. Inspecting the file using tar -tf confirmed there was no dump in there (looking at other full backups it is normally the first file in the archive). As it was a Sunday I couldn’t get in touch with Discourse, and we had promised to complete the migration by Monday morning, so I took a database only backup (around 7GB) and just used that to migrate.
Discourse are trying to help, but there is only really so much they can do now, as we’ve completed the migration and have been on our self-hosted environment since Sunday afternoon. Easiest solution would be that they just keep our S3 bucket and CDN active (for a fee) but they have said this is not possible. Reckon we are going to have to just lose the historic images tbh.
This is fixable. Just download the S3 bucket contents into your local uploads directory and then perform a remap on your database, rewriting the CDN and bucket URLs to the URL of your instance.
1 Like
Couple of issues - the size of the uploaded images would max out the SSD on our new VPS, and we don’t have the ability to attach an additional disk. Maybe we could just take a subset although not sure how this would work looking at the directory structure. Also we have already configured the site to use S3 for uploads instead of local storage.
Ok, then copy their S3 (or backup from S3) to your S3 and remap?
1 Like
Yeah, that is what I am hoping is possible!
1 Like
Ah. I see. Yeah once you’ve gone live you’re committed. It’s still quite possible to get the files moved to S3 before they are deleted.
You always needed to have enough space to hold all is the images to do the restore. You could copy the images one at a time. There are also tools that will copy the files directly, I believe.
Yeah the original plan was to use a temp Azure VM to restore, with large disk attached, then push back out to S3, take another backup once complete and finally move to our VPS (trying to keep costs down).
So I have a tar.gz containing all uploads and can get these directly into our S3 bucket maintaining the directory structure (think the standard AWS uploader can do it nowadays, else there is the CLI). Might be some consideration of ownership / permissions although maybe not.
After that it would be the remap - not sure of difference between discourse.remap and dbhelper.remap. I would look to test all of this on a dummy install with a couple of files first.
1 Like