Strange behavior with Digital Ocean Spaces/S3 and lots of duplicated CORS rules for `example.com`

installation
1

I got this error on a recent bootstrap. I’m using Digital Ocean in SFO3.

"Tasks: TOP => s3:upload_assets => s3:ensure_cors_rules", "(See full trace by running
 task with --trace)", "I, [2021-12-16T21:19:05.748497 #1]  INFO -- : Installing CORS rules...", "Attempting to apply ASSETS S3 CORS ruleset in bucket pfaffmanager.", "At
tempting to apply BACKUP_DIRECT_UPLOAD S3 CORS ruleset in bucket pfaffmanager."]

Is there something that could explain this that has a solution?

Hmm. I looked at the settings and there were a bunch of apparently identical CORS rules. I deleted one of them and it deleted all of them. This is a staging server that I have run a zillion (or maybe a hundred, anyway?) upgrades on, so maybe this would eventually happen to everyone using Digital Ocean, but for most people it would take more like 10 years than 8 months.

After deleting the CORS rules, the next bootstrap succeeded and there were then 2 CORS rules in the settings for the spaces bucket. But Look–it appears that the rules are set for example.com. :man_shrugging:

image
image628×702 30 KB

Another bucket in AMS3 that I am sure I’ve bootstraps Pretty Many times has 3 rules like the one above. All for example.com. I checked app.yml to see if I was doing something silly (and if I were then it would make sense I would have done it the same way in a bunch of places), but there’s no example.com in the yml.

On the rebuild, I see this in the logs:

                                                                                                                                                                         
I, [2021-12-16T21:46:58.038151 #1]  INFO -- : > cd /var/www/discourse && sudo -E -u discourse bundle exec rake s3:upload_assets                                          
I, [2021-12-16T21:47:39.039145 #1]  INFO -- : Installing CORS rules...                                                                                                   
Attempting to apply ASSETS S3 CORS ruleset in bucket pfaffmanager.                                                                                                       
Attempting to apply BACKUP_DIRECT_UPLOAD S3 CORS ruleset in bucket pfaffmanager.                                                                                         
Assets rules status: rules_applied.                                                 
Backup rules status: rules_applied.                                                                                                                                      
Direct upload rules status: rules_skipped_from_settings.

I looked in the code in places like s3_backup_store.rb for example.com and can’t come up with an explanation.

3 Likes
2

Looks like we should recommend DISCOURSE_S3_INSTALL_CORS_RULE: false for DO like we already do for GCP and BackBlaze in Using Object Storage for Uploads (S3 & Clones) :thinking:

4 Likes
3

Hello,

Sorry for revive this topic but I think I had a related issue few days ago.

My setup:

Digital Ocean Spaces with s3:upload_assets and s3:expire_missing_assets.
CDN - Bunny

It seems every rebuild or admin update when it upload assets generate a new CORS rule. It is actually duplicate the original rule. Unfortunately I didn’t copy the console error message but it was something before uploading assets - reached the maximum number 100 of CORS rules so I went to Digital Ocean and check the rules and yeah that was 100. :smiley: So I did the same like @pfaffman, I removed one and it removed all rules. After this the assets uploaded successfully on rebuild.

When I setted up this I followed this instruction here: Configure an S3 compatible object storage provider for uploads which is not contains this line yet.

Adding this line will stop generate duplicated CORS rules? Thank you :slightly_smiling_face:

4

I updated the other topic so hopefully it won’t happen to users in the future! Thanks for pointing this out.

2 Likes