Exported reports attachments (I tested with consolidated page views) are not marked as secure despite the file having a private acl, which prevents downloading as the short url pointed to an unsigned link. After running the uploads:secure_upload_analyse_and_update rake task it was properly marked as secure (there were 3 other posts/5 uploads also found but I was unable to determine what they were)
סיכום מבוסס בינה מלאכותית
Users are experiencing an issue where exported report attachments (ZIP files) are not marked as secure when secure media is enabled, preventing download. Wolftallemo reports that the uploads lack secure status despite having private ACLs, and running the uploads:secure_upload_analyse_and_update rake task resolves it. Martin notes that on a secure media site, uploads are correctly marked secure, and suspects the issue may relate to the secure_media setting only validating against AWS S3. Falco confirms this limitation. Martin observes that the security_last_changed_reason indicates
Can you clarify what you mean by “exported report attachment” ? Screenshot?
This is odd, if I try this on a secure media site the upload is correctly marked secure. Can you show the upload record like this after trying again?
#<Upload:0x0000556ae80c5208
id: 532362,
user_id: 1436,
original_filename: "consolidated-page-views-220318-031153-54.zip",
filesize: 480,
width: nil,
height: nil,
url: "//blah.zip",
created_at: Fri, 18 Mar 2022 03:11:53.556489000 UTC +00:00,
updated_at: Fri, 18 Mar 2022 03:11:53.842038000 UTC +00:00,
sha1: "12345",
origin: nil,
retain_hours: nil,
extension: "zip",
thumbnail_width: nil,
thumbnail_height: nil,
etag: "12345",
secure: true,
access_control_post_id: 377702,
original_sha1: "12345",
verification_status: 1,
animated: nil,
security_last_changed_at: Fri, 18 Mar 2022 03:11:53.836860000 UTC +00:00,
security_last_changed_reason: "login is required | source: post creator">
Is your site login required?
It is not login required
#<Upload:0x000055646d495a30
id: 62749,
user_id: 1,
original_filename: "web-crawlers-220318-032906-26.zip",
filesize: 3017,
width: nil,
height: nil,
url:
"//[nope].storage.googleapis.com/original/3X/6/7/679649f9c6d33541cf5f5d2c48c2ef514bde36a0.zip",
created_at: Fri, 18 Mar 2022 03:29:07.114686000 UTC +00:00,
updated_at: Fri, 18 Mar 2022 03:29:07.328592000 UTC +00:00,
sha1: "679649f9c6d33541cf5f5d2c48c2ef514bde36a0",
origin: nil,
retain_hours: nil,
extension: "zip",
thumbnail_width: nil,
thumbnail_height: nil,
etag: "54f0df6d95a84d04877aa20f238c3b1e",
secure: false,
access_control_post_id: 214238,
original_sha1: "5cc4f437505ae3a07bdd27bbe2653462de31db6d",
verification_status: 1,
animated: nil,
security_last_changed_at: Fri, 18 Mar 2022 03:29:07.112534000 UTC +00:00,
security_last_changed_reason: "no checks satisfied | source: upload creator">
Our secure_media site setting is only validated against AWS S3. That may be the issue.
This is the strange part:
security_last_changed_reason: "no checks satisfied | source: upload creator"
For me, with login_required false and secure_media true in my site settings, I get this when I export a report and it gets PM’d to me:
security_last_changed_reason: "access control post dictates security | source: post creator"
This makes sense because the post creator for the PM has the upload attached, and at that point it should be set to secure: true. You have an access_control_post_id on that upload record but it doesn’t seem to have worked correctly?
What happens if you do Post.find(214238).with_secure_media?
I don’t think that should affect it, this would only affect ACLs I think.
Would this not apply to all potentially secure uploads? Considering that posts made in private topics and other PMs don’t have this issue, I’m not certain about that.
=> true
Hmm… Not sure what happened here then
So strange, if I add a breakpoint inside PostCreator (which is called from the export job) I get a similar result to yours at first for the upload:
secure: false,
access_control_post_id: 67115,
...
security_last_changed_at: Fri, 18 Mar 2022 04:14:42.292485000 UTC +00:00,
security_last_changed_reason: "no checks satisfied | source: upload creator"
But then as soon as the PostCreator update of the secure status occurs everything is fine:
secure: true,
access_control_post_id: 67115,
...
security_last_changed_at: Fri, 18 Mar 2022 04:14:55.645303000 UTC +00:00,
security_last_changed_reason: "access control post dictates security | source: post creator"
Does Discourse.store.external? return true for you?
def update_uploads_secure_status(source:)
if Discourse.store.external?
Jobs.enqueue(:update_post_uploads_secure_status, post_id: self.id, source: source)
end
end
It does, I don’t see any running or scheduled jobs in sidekiq so I assume it either failed or never ran.
I am so confused
Anything in your /logs page that looks likely related to this? It seems like the only way this could be happening is if that update_post_uploads_secure_status sidekiq job is failing or erroring in some way.
There was a few errors but they were all related to the CleanUpUploads job. Upon further investigation it seems the job never ran (there was no failed jobs in the last 2 days)
I’m sorry, I’m not able to reproduce this so there’s not a whole lot else we can do with this for now.
