Strange behavior with Digital Ocean Spaces/S3 and lots of duplicated CORS rules for `example.com`

Product Support Self-hosting

AI-generated summary

Users report a recurring issue with Digital Ocean Spaces/S3 where repeated Discourse bootstraps or rebuilds generate duplicate CORS rules—often for example.com—eventually hitting the 100-rule limit and causing failures during asset uploads. pfaffman first encountered this on a staging server, noting that deleting one rule removed all, and observed that post-cleanup, only two rules remained—still targeting example.com, despite no such setting in app.yml. Falco suggests setting DISCOURSE_S3_INSTALL_CORS_RULE: false to prevent automatic rule creation, as already recommended for GCP and BackBlaze. Don confirms a similar experience with DO Spaces + BunnyCDN, hitting the 100-rule cap during rebuilds, and successfully resolved it by deleting all rules manually before adding the suggested config. pfaffman confirms the fix and updates documentation to help future users avoid the issue. The root cause appears tied to Discourse’s automatic CORS rule installation logic, which may not deduplicate properly on DO Spaces, leading to accumulation over time.

1

I got this error on a recent bootstrap. I’m using Digital Ocean in SFO3.

"Tasks: TOP => s3:upload_assets => s3:ensure_cors_rules", "(See full trace by running
 task with --trace)", "I, [2021-12-16T21:19:05.748497 #1]  INFO -- : Installing CORS rules...", "Attempting to apply ASSETS S3 CORS ruleset in bucket pfaffmanager.", "At
tempting to apply BACKUP_DIRECT_UPLOAD S3 CORS ruleset in bucket pfaffmanager."]

Is there something that could explain this that has a solution?

Hmm. I looked at the settings and there were a bunch of apparently identical CORS rules. I deleted one of them and it deleted all of them. This is a staging server that I have run a zillion (or maybe a hundred, anyway?) upgrades on, so maybe this would eventually happen to everyone using Digital Ocean, but for most people it would take more like 10 years than 8 months.

After deleting the CORS rules, the next bootstrap succeeded and there were then 2 CORS rules in the settings for the spaces bucket. But Look–it appears that the rules are set for example.com. :man_shrugging:

image
image628×702 30 KB

Another bucket in AMS3 that I am sure I’ve bootstraps Pretty Many times has 3 rules like the one above. All for example.com. I checked app.yml to see if I was doing something silly (and if I were then it would make sense I would have done it the same way in a bunch of places), but there’s no example.com in the yml.

On the rebuild, I see this in the logs:

                                                                                                                                                                         
I, [2021-12-16T21:46:58.038151 #1]  INFO -- : > cd /var/www/discourse && sudo -E -u discourse bundle exec rake s3:upload_assets                                          
I, [2021-12-16T21:47:39.039145 #1]  INFO -- : Installing CORS rules...                                                                                                   
Attempting to apply ASSETS S3 CORS ruleset in bucket pfaffmanager.                                                                                                       
Attempting to apply BACKUP_DIRECT_UPLOAD S3 CORS ruleset in bucket pfaffmanager.                                                                                         
Assets rules status: rules_applied.                                                 
Backup rules status: rules_applied.                                                                                                                                      
Direct upload rules status: rules_skipped_from_settings.

I looked in the code in places like s3_backup_store.rb for example.com and can’t come up with an explanation.

2

Looks like we should recommend DISCOURSE_S3_INSTALL_CORS_RULE: false for DO like we already do for GCP and BackBlaze in Using Object Storage for Uploads (S3 & Clones) :thinking:

3

Hello,

Sorry for revive this topic but I think I had a related issue few days ago.

My setup:

Digital Ocean Spaces with s3:upload_assets and s3:expire_missing_assets.
CDN - Bunny

It seems every rebuild or admin update when it upload assets generate a new CORS rule. It is actually duplicate the original rule. Unfortunately I didn’t copy the console error message but it was something before uploading assets - reached the maximum number 100 of CORS rules so I went to Digital Ocean and check the rules and yeah that was 100. :smiley: So I did the same like @pfaffman, I removed one and it removed all rules. After this the assets uploaded successfully on rebuild.

When I setted up this I followed this instruction here: Configure an S3 compatible object storage provider for uploads which is not contains this line yet.

Adding this line will stop generate duplicated CORS rules? Thank you :slightly_smiling_face:

4

I updated the other topic so hopefully it won’t happen to users in the future! Thanks for pointing this out.