Download backup - email link only

[Kevin_McKinney]

In the past when I clicked the Download button on the backups screen, the browser downloaded the file. Now the tooltip reads “Send email with download link” and it emails me with a link to download it. Is that new behavior that I just missed or is there an issue with my install?

[jomaxro (Joshua Rosenfeld)]

Yes, that is new behavior added to increase the security of the backups. I’m looking for the relevant topic now…

Edit: Here you go:

Security breach in Discourse forum with SSO

Email confirmation of backup downloads is now in, thanks to @blake. You can’t download backups simply by being logged in, you must… Be logged in as an admin → to get to /admin/backups Control the email address of an admin → to receive the download link email So this is indeed a form of two factor auth. Verifies control of the email account, plus the ability to log in as the admin. Here’s the actual email the admin will receive when the download backup button is pressed; it warns about unknow…

[ErikMH (Erik Mueller-Harder)]

So. I have four Discourse forums. I like to download my backups daily, for each. In each case, I must navigate to the “backups” page, click on the download link, switch over to my e-mail program, wait for the message to arrive, and click on the link that’s provided? Every day?

I understand the need for security (and that’s one reason I don’t use SSO at all!), but this sucks.

Given this convoluted workflow, is there some way that I could easily bypass all of this and automatically download the most recent backup for each of my sites daily?

[eviltrout (Robin Ward)]

Have you considered having your uploads automatically backup to S3? If you want to take backups every day that’s even simpler than logging in to download them.

[ErikMH (Erik Mueller-Harder)]

Thanks, @eviltrout, I may look into that. I’ve usually stayed away from Amazon’s services, and so had forgotten that this was a baked-in option.

(Logging in hasn’t been an issue, as I’m a moderator on each of the forums anyway, and have browser tabs for them pinned open — so downloading the automatic backups been dead easy.)

[eviltrout (Robin Ward)]

We are also going to be adding more cloud providers for automatic backups in the future via the Rails Girls Summer of Code project, if Amazon isn’t your thing.

The best kinds of backups are the ones you don’t have to do anything about

[Falco]

@eviltrout

We are also going to be adding more cloud providers for automatic backups in the future via the Rails Girls Summer of Code project, if Amazon isn’t your thing.

Yeah, and Dropbox Backups already works.

[steef]

Is there a way to disable this feature? I used to download backups automatically to a server in my control by inspecting /admin/backups.json using an admin API token (of an account which doesn’t have an email address in the first place).

I’d be more interested in being able to set an encryption key for the backups. If someone manages to steal the API key for my backup bot then I could still rest relatively easy knowing the private key is only on the backup download server, which doesn’t face the internet apart from downloading the backups.

I understand the security implications, however, I’d consider storing (unencrypted) backups on someone else’s server to be more dangerous than the risk of someone obtaining the API key for my bot (which resides only on a system I have physical control of, and does not act as a server). Making Discourse push backups to the system would be less optimal for me as that would mean I had two internet-facing servers to worry about instead of one.

[eviltrout (Robin Ward)]

There’s no way to disable this feature, and because we want to be very secure by default I can’t see us allowing a user to do that any time soon. You could build a plugin that allows you to access the backups directly.

The encryption key is not a bad idea as a separate feature, but it wouldn’t mean we’d pull back this feature.

[steef]

Sorry for the late reply.

Oh yeah, allowing a user to disable the feature kind of makes the feature meaningless I suppose, hadn’t thought of it that way.

Then I’ll consider making a plugin .Thanks for the quick response and suggestion!

[michaeld (Michael - Communiteq)]

We had an issue here because it made it much harder to download a backup from a (third party) provider where we don’t have shell access, and upload it to one of our own servers. Especially when handling a large backup and both servers are in the USA (we’re in Europe) this process was costing us hours instead of minutes.

Somehow the backup link does not work with an api_key so a simple wget will not work

We made a small script that can be called from the command line. It needs three parameters:

• username
• password
• download link from the email, including the token

So something like

./download.sh myuser mypass https://forum.mysite.com/admin/backups/mysite-2017-05-31-113000-v20170526125321.tar.gz?token=d31de61bc07f7bc9060717bc57828584

Here is the script:

#!/bin/bash
USERNAME=$1
PASSWORD=$2
URL=$3
HOSTNAME=$(echo $URL | awk -F/ '{print $3}')

TOKEN=$(
curl \
  --header "X-Requested-With: XMLHttpRequest" \
  -A "MSIE" \
  -b cookies.txt \
  -c cookies.txt \
  -v\
  https://$HOSTNAME/session/csrf|awk -F: '{print $2}'|cut -c2-89
)

echo "Using Token $TOKEN Username $USERNAME Password $PASSWORd"
curl \
  --header "X-Requested-With: XMLHttpRequest" \
  --header "X-CSRF-Token: $TOKEN" \
  --header "Origin: https://$HOSTNAME"  \
  -A "MSIE" \
  -b cookies.txt \
  -c cookies.txt \
  -v\
  --data "login=$USERNAME&password=$PASSWORD" \
  https://$HOSTNAME/session

curl \
  -A "MSIE" \
  -b cookies.txt \
  -c cookies.txt \
  -O \
  $URL

One more suggestion: would it be an idea to exclude the email with the link from being affected by the disable_email setting?

[xiasummer]

I think it a quite bad idea.

Because the browser downloader usually stops at some time. The backup package is quite big usually, but if we only have one chance to download and then the link will pastdue, usually it will download half and stoped there.

Can you support breakpoint resume download? If not, you’d better think of a better way to help people successfully download the whole package with the browser.

[C_T]

I have the same issue. I have to disable this feature for personal reasons.

[C_T]

I understand that you want to be very secure by default. But can we have non-default options?
Or can we have an option to send the link to a new target email address?
I strongly agree to set it to enable as a default option.

[C_T]

There are many features that can be set as disable or not default. Maybe they are not supposed to be meaningless.

[Jagster (Jakke Lehtonen)]

Email link or not isn’t the main issue here. Now we must remember that downloading backup was a solution for real issue and that was, and is, too rarely done backups.

I take backup of database of my webstore every 5 minutes. My CMS takes backup every hour. With Discourse the freshest fish I get is once a day natively. That means because I’m lost with docker and anything else another than MySQL/MariaDB I’m taking everyday a risk to loose some content if something goes bad in this window of 24 hours.

That has happend to me once, actually.

[C_T]

With the current email link method, when I was downloading backups in a region with a restricted network, I always got stuck at 30%, even if I used VPN for some time.

[C_T]

I will have a similar issue in the future if it is still email-link-only at that time.

[pfaffman (Jay Pfaffman)]

@C_T

I understand that you want to be very secure by default. But can we have non-default options?

If you are self-hosted and want to download the backup you can do it via ssh/scp. Another option is to have backups stored on S3. Downloading them via the web browser is not a reliable way to automate the process.

@Jagster

I take backup of database of my webstore every 5 minutes.

If you want that level of being able to go back in time then you should configure postgres to allow rollbacks and/or have a live backup server.