Cannot download exported reports when secure media is enabled

Bug
1

Exported reports attachments (I tested with consolidated page views) are not marked as secure despite the file having a private acl, which prevents downloading as the short url pointed to an unsigned link. After running the uploads:secure_upload_analyse_and_update rake task it was properly marked as secure (there were 3 other posts/5 uploads also found but I was unable to determine what they were)

1 Like
4

Can you clarify what you mean by “exported report attachment” ? Screenshot?

1 Like
5

Sorry about that, I’m referring to the ZIP file linked in the pm sent as the export is finished.
image

1 Like
7

This is odd, if I try this on a secure media site the upload is correctly marked secure. Can you show the upload record like this after trying again?

#<Upload:0x0000556ae80c5208
 id: 532362,
 user_id: 1436,
 original_filename: "consolidated-page-views-220318-031153-54.zip",
 filesize: 480,
 width: nil,
 height: nil,
 url: "//blah.zip",
 created_at: Fri, 18 Mar 2022 03:11:53.556489000 UTC +00:00,
 updated_at: Fri, 18 Mar 2022 03:11:53.842038000 UTC +00:00,
 sha1: "12345",
 origin: nil,
 retain_hours: nil,
 extension: "zip",
 thumbnail_width: nil,
 thumbnail_height: nil,
 etag: "12345",
 secure: true,
 access_control_post_id: 377702,
 original_sha1: "12345",
 verification_status: 1,
 animated: nil,
 security_last_changed_at: Fri, 18 Mar 2022 03:11:53.836860000 UTC +00:00,
 security_last_changed_reason: "login is required | source: post creator">

Is your site login required?

1 Like
8

It is not login required

#<Upload:0x000055646d495a30
 id: 62749,
 user_id: 1,
 original_filename: "web-crawlers-220318-032906-26.zip",
 filesize: 3017,
 width: nil,
 height: nil,
 url:
  "//[nope].storage.googleapis.com/original/3X/6/7/679649f9c6d33541cf5f5d2c48c2ef514bde36a0.zip",
 created_at: Fri, 18 Mar 2022 03:29:07.114686000 UTC +00:00,
 updated_at: Fri, 18 Mar 2022 03:29:07.328592000 UTC +00:00,
 sha1: "679649f9c6d33541cf5f5d2c48c2ef514bde36a0",
 origin: nil,
 retain_hours: nil,
 extension: "zip",
 thumbnail_width: nil,
 thumbnail_height: nil,
 etag: "54f0df6d95a84d04877aa20f238c3b1e",
 secure: false,
 access_control_post_id: 214238,
 original_sha1: "5cc4f437505ae3a07bdd27bbe2653462de31db6d",
 verification_status: 1,
 animated: nil,
 security_last_changed_at: Fri, 18 Mar 2022 03:29:07.112534000 UTC +00:00,
 security_last_changed_reason: "no checks satisfied | source: upload creator">
9

Our secure_media site setting is only validated against AWS S3. That may be the issue.

1 Like
10

This is the strange part:

security_last_changed_reason: "no checks satisfied | source: upload creator"

For me, with login_required false and secure_media true in my site settings, I get this when I export a report and it gets PM’d to me:

 security_last_changed_reason: "access control post dictates security | source: post creator"

This makes sense because the post creator for the PM has the upload attached, and at that point it should be set to secure: true. You have an access_control_post_id on that upload record but it doesn’t seem to have worked correctly?

What happens if you do Post.find(214238).with_secure_media?

I don’t think that should affect it, this would only affect ACLs I think.

1 Like
11

Would this not apply to all potentially secure uploads? Considering that posts made in private topics and other PMs don’t have this issue, I’m not certain about that.

12

=> true

Hmm… Not sure what happened here then

13

So strange, if I add a breakpoint inside PostCreator (which is called from the export job) I get a similar result to yours at first for the upload:

  secure: false,
  access_control_post_id: 67115,
...
  security_last_changed_at: Fri, 18 Mar 2022 04:14:42.292485000 UTC +00:00,
  security_last_changed_reason: "no checks satisfied | source: upload creator"

But then as soon as the PostCreator update of the secure status occurs everything is fine:

 secure: true,
 access_control_post_id: 67115,
...
 security_last_changed_at: Fri, 18 Mar 2022 04:14:55.645303000 UTC +00:00,
 security_last_changed_reason: "access control post dictates security | source: post creator"

Does Discourse.store.external? return true for you?

  def update_uploads_secure_status(source:)
    if Discourse.store.external?
      Jobs.enqueue(:update_post_uploads_secure_status, post_id: self.id, source: source)
    end
  end
14

It does, I don’t see any running or scheduled jobs in sidekiq so I assume it either failed or never ran.

15

I am so confused :thinking: Anything in your /logs page that looks likely related to this? It seems like the only way this could be happening is if that update_post_uploads_secure_status sidekiq job is failing or erroring in some way.

16

There was a few errors but they were all related to the CleanUpUploads job. Upon further investigation it seems the job never ran (there was no failed jobs in the last 2 days)

19

I’m sorry, I’m not able to reproduce this so there’s not a whole lot else we can do with this for now.

1 Like