Errors on Exporting Data from Teams to Self Hosted Discourse on Digital Ocean

Hi folks!
Ran into some data export-restore kind of issues and would request your help.

Context:
Wanted to export data from Discourse Teams to a self-hosted droplet on digital ocean.
Backup, restore went smoothly. No issues. But post the restore, here’s what discourse shows:


https://dsh.re/d3f85 (here’s a screen recording - sorry this forum didn’t allow me to upload it natively)

I did try to restore once again. same issue.

Here’s the data restore log in txt:
https://dsh.re/311b8

Please help.

Edit:
Here is the screenshot from /logs
dsh.re/64209a (please add https:// - sorry forum didn’t let me upload image / add another url)

3 Likes

Hi Yash! Sorry to hear you’re having some trouble with the backup/restore. Have you tried safe mode yet? Just add /safe-mode to the end of the URL. You can then see what hapens when you disable the current theme, unofficial plugins or all plugins.

I also recommend you take a look at /logs and the nginx error log to see if there are any errors that might help you further.

And what happens when you run ./launcher rebuild app from the command line? You may want to try running it multiple times. This has worked for me in the past.

And finally, can you share the contents of your containers/app.yml file (without passwords and site identifying info please) so we can see what plugins you have installed.

4 Likes

Hey Tobias,
Thank you so much for the prompt reply. Genuinely appreciate your help.

  1. When i go to /safe-mode it gives me a few checkbox options such as disable theme etc. Once I check all of them and click next, it’s the same error on the frontend.

The url appended was /?safe_mode=no_custom%2Cno_plugins%2Conly_official

Please note, this is the one-click DO droplet with no additional plugins installed by me yet.
But the Teams instance had a lot of plugins. Do you reckon that’s causing the issue?

  1. Here are the /logs
    https://dsh.re/64209a

  2. Trying ./launcher rebuild app right now. The first attempt yielded no result. Trying again. Same error on the frontend after logging in.

  3. Here’s a snapshot of the app.yml file. Should I export the file and send it here?

Any thoughts on what’s going wrong?

2 Likes

Wait. The only supported install method is this one: discourse/INSTALL-cloud.md at master · discourse/discourse · GitHub

Please try that and then let us know how you get on.

2 Likes

https://marketplace.digitalocean.com/apps/discourse - this won’t work? I’ll have to delete the droplet and re-setup using the instructions you have shared. Is my understanding correct?

2 Likes

That’s right. Sorry about that! You’ll have to start over.

2 Likes

Okay, will try this and post the results here. Thank you Tobias!

3 Likes

Hey @tobiaseigen
I setup a new droplet, installed discourse following the instructions on github. Logged in, uploaded the backup, restore - same error.

Restore log:
https://dsh.re/05eb5

Error log at /logs

App.yml

Could you please help me with the next steps? I’ve tried launcher rebuild and safe mode too. Same error. No plugins have been installed by me so far.

Side note - the DO one click installer uses docker and the latest release from github. Won’t it be exactly the same as a manual install?

4 Likes

Ok great we are a step closer. I will look at this next time I am at a computer.

4 Likes

It’s an issue with uploads. It’s looking for an S3 bucket name but the new environment is not configured with one.

How was the backup created and are all uploads included? I don’t see any uploads happening in the restore log, which suggests there were none in the backup.

3 Likes

Ah! Was this site terminated before you made the backup? If not, the assets will be on S3, so a full-restore will fail.

In order for the backup to work correctly, a hidden setting must be turned on to have all of the S3 assets included in the backup.

It’s conceivable that canceling a teams site doesn’t automatically trigger this setting getting turned on.

3 Likes

I am fairly certain this is the usual backup download that occurs when a site is decomissioned. It includes all uploads.

I’m going to test this out for myself tomorrow and confirm that the backup/restore process still works as it should on Teams.

3 Likes

I just did a quick test of the backup process and see that the uploads are indeed included in the backup file. Tomorrow I will see if I can figure out how to restore this to a fresh self-hosted site.

2 Likes

Hi @schleifer - As Tobias rightly pointed out, the backup was created from teams (the usual backup download). When i clicked on cancel subscription, it gave me an option to backup and download that backup. Post that I have taken a separate backup from the admin panel. Both have the same file size, and neither of them work on the self-hosted install.

@pfaffman - No no, the site was still active when I made the backup. I’m not sure about the hidden setting. Maybe that could be the issue.

@tobiaseigen - When i unzip the tar.gz file, the only content is “dump.sql”. I don’t see any media/ attachments. Did I do something wrong when I took the backup?

3 Likes

Yeah. That’s not the backup you need. If you have only the sql dump then you’re going to need to check with them and get the backup with the uploads. I’m pretty sure they’ll get it sorted out tomorrow and you’ll be all good.

3 Likes

Usually it will (of course) miss the uploads but restore will not fail on that?

4 Likes

I had the same thoughts - if the files are missing, shouldn’t discourse still show all the threads, external links and conversations without the natively uploaded attachments?

3 Likes

I think that if he pulls out the dump.sql.gz and renames it with the full-dump-filename.sql.gz it’ll restore but if it had the tar.gz name it tries to process uploads and fails. (Maybe there is S3 stuff in the site settings? I don’t quite get it.) There was a similar topic last week and that was Gerhard’s recommendation.

But if he really wants to host the site, he wants the uploads, so he might as well wait for the full dump.

3 Likes

@biyaniyash can you please go to /admin/site_settings/category/all_results?filter=s3 on your new site and disable everything related to s3, then give it another shot? That should get your site working.

1 Like

Hey @tobiaseigen:
This is what I see by default when I go to /admin/site_settings/category/all_results?filter=s3 on the new self-hosted discourse site:

All but one option seems to be disabled by default. Could you confirm what are the steps I need to take?

3 Likes