Download a backup from an email link with `wget`

Wiki Sysadmins
,

AI-generated summary

This discussion explores downloading Discourse backups via wget when SSH access isn’ a viable option. pfaffman details a method of copying the download URL from Chrome (after initiating a download through the Discourse interface) and using it with wget to transfer the backup file to another server, noting the time-limited nature of the URL and steps for restoration. The post includes an example wget command and instructions for restoring the backup via the command line.

However, the method’s effectiveness is questioned. darkpixlz reports it doesn’. work for sites not using S3, sharing an error message, and pangbo explains that Discourse typically uses cookie-based authentication which wget doesn’t automatically handle, whereas the provided URL relies on S3 signature authentication. pfaffman acknowledges this and suggests they may have only ever used the method with S3-backed sites (like those from CDCK). pfaffman advises quoting the URL when using it with wget to avoid issues with the & character.

1

For a seasoned sysadmin, the favored way of moving large files is a tool like rsync or scp, but sometimes, you don’t have ssh access to the host where you need to retrieve a large backup file.

Because backup files contain sensitive information, Discourse’s has security features that make it very difficult for an unauthorized person to retrieve a backup. You must get a download link via email and use that link logged in as the user who requested it. If you want to download the data with a web browser, it’s quite painless. If you want to move that backup to another server on the internet, most residential internet service makes that a very painful proposition. On my home internet connection a 2.3 GB file takes over 20 minutes to upload. The 12GB file here would take on the order of two hours.

Here is how to get a link that will allow you to pull a backup from a Discourse site via wget.

First, initiate the download as usual and then open Chrome’s downloads page (chrome://downloads/). Right-click on the URL and copy it.

download-dialog
download-dialog715×203 16.3 KB

Then, in a shell on the machine where you want the file, you can then paste that URL into a wget request. Make sure that you put quotes around it as the &s will cause you problems. You also need to make sure to maintain the backup file’s original filename. Your request should look something like this:

wget --show-progress "https://bucket-name.s3.us-west-2.amazonaws.com/backups/xyz/multisitename/discourse-2020-11-19-001538-v20201116132948.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWWK5WHOFJ%2F20201119%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20201119T013442Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=1753b97a8aaf6953c89aa28628b8db" -o discourse-2020-11-19-001538-v20201116132948.tar.gz

You can also omit the -o filename and rename the file (which will have the full URL as its filename) after you’ve downloaded it. The URL is time-limited, so you’ll need to initiate the download when you’re ready to pull it to your server.

You can then move the file to /var/discourse/shared/standalone/backups/default and restore it from the web interface or with

cd /var/discourse
./launcher enter app
discourse enable_restore
discourse restore

The final command above will print a list of the available backups and you can copy-paste the correct one to start the restore.

14 Likes
Sysadmins Index
Can you get the token needed to download backups via the API?
2

Sadly this does not seem to work outside of sites using S3.

image
image1024×186 34.6 KB

3

I believe that it will work.

1 Like
4

I think this won’t work, as S3 authenticates identity through the X-Amz-Signature parameter in the URL, whereas downloading directly from Discourse requires cookie-based authentication. Merely copying the URL into wget does not suffice for authentication.

1 Like
5

Hmm. Maybe that’s right. Maybe I’ve only ever done it via S3 (like probably from CDCK sites where I don’t have ssh access).