Secure Media Uploads

Please read the topic, your issue is addressed a few posts above this one.

3 Likes

I read everything carefully. Please let me know what exactly you mean by few posts above. I don’t see this issue.
Edit:
Trying to read between the lines here, i see that this thread used to be longer but some key replies were deleted from it, i see references to replies that don’t exist and mentions of screenshots that don’t appear in the thread.
If I understand correctly though, the recommendation is to set S3 bucket to NOT block any public access at all. I just want to confirm that, and ask if it is safe?

1 Like

Correct.

The original post is indeed gone but the problem and solution are still there

6 Likes

I noticed a bug with using secure media and Knowledge Base Plugin
Links to attachments in the knowledge base fail to open (redirect to 404 page) unless forced to be opened in a new window.
The extra weird thing is that the very same attachment can be opened without issue from the Discourse topic associated with the knowledge base item.

Edit:
The same bug happens when someone is copying a link to an attachment from one reply to another. the link is the exact same one naturally, but it only works from the original reply, unless forced to be opened in a new window.

1 Like

What about secure media if s3 is not used?
Currently it looks like that, if you have the direct link to the uploaded file, you can download that file without login. Which is like a security issue then…

Secure media requires s3.
This entire feature was developed to make it impossible to share and access direct links.

10 Likes

This is an excellent feature. Thank you for developing it. Minio, a free drop-in replacement for S3, has no support for ACLs and never will. Their argument, which is a good one, is that ACL are not a useful feature and negatively affect performance because they require a second write operation. Secure media uploads will work with Discourse - the uploads:secure_upload_analyse_and_update task will error out on the final step but it seems you can ignore it. That said, is there any reason for Discourse to make ACL calls at all?

2 Likes

Yes, because the S3 bucket is private based on the setup in the OP, any non-secure uploads need to have their ACL set to public, and secure uploads will have a private ACL so the S3 URL can be accessed directly. I do not think we have plans at this time to modify how this works; I think there would be quite a bit of work involved to avoid using ACLs altogether for S3 replacements.

7 Likes

For a completely private forum (no anonymous access), the lack of a secure media option in the default local storage configuration seems like a huge oversight to me. I wouldn’t be surprised if a lot of folks running private forums don’t realize that all of their post attachments are publicly accessible (if you know the URL). If my post’s text can’t be accessed anonymously, intuitively I would think the post’s attachments can’t be either.

Is it worth submitting an issue requesting this to see what sort of support it would get? Does one already exist? This one issue is currently a deal breaker for my small, poor group who is trying to spin up Discourse in an environment where we own the hosting hardware already and don’t want to pay for cloud storage (particularly at AWS S3 prices) just to make sure our media isn’t viewable anonymously.

Also, forgive the potential naivete of this question. But there is already support for distinct buckets for uploads and backups, right? Would it be difficult to support a third bucket for secure uploads? Public uploads go in the public bucket, secure uploads go in the secure bucket, and no individual object ACLS are needed. And if the security status of an object changes, it could just be moved between buckets?

2 Likes

I think it’s quite a bit of work. At least a day’s work, but maybe a week?

Cdck hosting, which pays for development, is run with uploads on S3, so it’s only login required self hosted sites without budget for S3 that’s interested.

If your users see sharing links to your assets they could just as well be downloading them to share them. It doesn’t seem like a big problem.

3 Likes

I’m not crazy pants for thinking it’s weird that post text is protected but attachments are not though, right?

There’s a big difference between intentional and accidental sharing though. Obviously, there’s no real way to prevent the former. But the latter can happen right now simply because people don’t understand the underlying tech. Consider email notifications for posts that include links to attached images getting innocently forwarded to non-members. An option to redact attachments in emails that’s not tied to the secure media functionality might be a good alternative there.

Even when people do intentionally share attachment links, they might have simply forgotten that it was from a protected category. But in an ideal world, the attachment link should be worthless to anyone without access to that category.

A current or future bug in one of the S3 implementations could also potentially allow for anonymous enumeration of the resources in a bucket or the ability to guess and brute force object URLs. In an ideal world, I’d want my on-prem S3 interface to be firewalled off from the internet at large so only the Discourse server can reach it directly.

3 Likes

Not crazy, implementing secure uploads for non s3 setups is certainly not something I am against building. It is just that we have no urgency or push to build the feature and the change is non trivial.

We occasionally have interns and audition projects, this is certainly a type of project that could fit into one of those buckets.

9 Likes

Any thoughts on the feasibility of this as an easier path for secure media on not-AWS-S3 storage?

1 Like

I’ve got a quick hack that I think would provide secure uploads for login-required sites.

Basically, you set up NGINX Docs | Authentication Based on Subrequest Result for uploads. The only issue is that I can’t find a URL that returns a 403/401 when login-required is turned on, so accessing an upload when you’re not logged in gives a 500 error. This would only happen if someone had an upload URL and tried to access it when they weren’t logged in, so it doesn’t seem that bad.

It’s something like this:

# JP
    location = /auth {
        internal;
        proxy_pass http://discourse/categories;
        proxy_pass_request_body off;
        proxy_set_header Content-Length "";
        proxy_set_header X-Original-URI $request_uri;
    } 
    # END JP
    location ~ ^/uploads/ {

      auth_request /auth; #$JP
      # NOTE: it is really annoying that we can't just define headers
      # at the top level and inherit.
      #
2 Likes

We have no plans to make distinct bucket functionality for secure uploads, nor do we have plans to make secure uploads work with non-S3 setups or setups without ACLs. We may look at doing this in the future if there is sufficient demand for this work to a point where it makes sense for us to devote time and resources to this effort.

5 Likes

why its using https://hashhackersforum.s3.us-east-2.amazonaws.com/optimized/2X/3/3204e85df407adfce19e105308248aee8b3b3f57_2_690x424.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAU5WBGG5Z7FNHQEIS%2F20210415%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20210415T025617Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=15c9118929ccd9c24a9594ab02a47e900269b9d921d41679a317e29d6174c2bc
instead of https://forum.cdn.hashhackers.com/optimized/2X/3/3204e85df407adfce19e105308248aee8b3b3f57_2_690x424.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAU5WBGG5Z7FNHQEIS%2F20210415%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20210415T025617Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=15c9118929ccd9c24a9594ab02a47e900269b9d921d41679a317e29d6174c2bc

while i’ve entered CDN URL correctly.

It won’t use a CDN if you enable “Secure Media Uploads”.

5 Likes

One thing I’ve noticed is that the admin section warns that

The server is configured to upload files to S3, but there is no S3 CDN configured. This can lead to expensive S3 costs and slower site performance. See “Using Object Storage for Uploads” to learn more.

I don’t mind ignoring it, but we have secure media enabled which means it’s not possible to use a CDN. This message probably shouldn’t be shown on sites with secure_media enabled.

1 Like

A CDN can still be used for all the JS assets, making your site a lot faster to render for users around the world.

3 Likes

That’s cool. It wasn’t clear to me that discourse would treat the static assets differently. I think I need to read all the threads here again.

2 Likes