High usage of S3 head_bucket requests during initial migration

We switched our forum to store uploads and backups with Backblaze B2. We went through the S3 migration and rebake (which I left overnight to complete). Everything works.

B2 allows free 2500 class C transactions per day, which head_bucket falls under (s3_head_bucket). We hit that limit, with 2300 being head_bucket with only 5300 put_object requests (s3_put_object).

I want to understand better how Discourse uses head_bucket for S3 storage. Based on the docs, it’s used to check if a bucket exists. But 2300 requests seem a bit high to continuously check if buckets exist.

I would appreciate any information about head_bucket usage in Discourse. Thanks.


Hopefully, this post will also be a reference for anyone seeking this information. If anyone wants to use B2 free plan, they will hit the 2500 class C transactions daily cap and won’t be able to perform any more requests until the daily limit is reset or the payment method is added.

2 Likes

Since the original post, we have 5500 put_object requests and 2900 head_bucket. Still seems awfully high for no apparent reason.