S3 buckets containing periods not allowed

So, with the introduction of the s3_cdn_url setting, you can set a custom URL to be used instead of the long S3 URL.

Unfortunately, s3_upload_bucket doesn’t allow periods in the name… which means that if I, as per Amazon’s documentation, create a bucket called static.example.com and point a CNAME to it, I can set Discourse to print out URLs to static.example.com, but I can’t set it to actually upload to that bucket.

(I want to put Cloudflare in front of a CNAME, making for a low-cost but effective CDN. It’s not just for looks.)

2 个赞

If periods are indeed supported we should relax the restriction … @techAPJ can you have a quick look

As I understood it, periods in the bucket name cause havoc with https and certificates so it is strongly discouraged that you do this. Unless you never plan to use https.

http://shlomoswidler.com/2009/08/amazon-s3-gotcha-using-virtual-host.html

We ran into many problems with this, which is why it is disallowed. It is not random. It’s a resolution to a problem many Discourse instances faced.

5 个赞

aha … well then let’s just leave it as is for now.

1 个赞

This is a good point. However, I think it should be downgraded to a warning, rather than being outright forbidden.

While locking yourself out of HTTPS is a terrible idea on its own, both Amazon CloudFront and CloudFlare (off the top of my head) offer SNI-based SSL certificates that can be put in front of a bucket, even if the default certificate would no longer cover it.

CloudFront can be configured to pull from an S3 bucket of your choice (even if it’s named something different), but also costs money. Well invested money, one may argue, but money nonetheless. If one does not need the additional functionality of a full-fledged CDN, CloudFront is a slightly overkill solution to the problem of “I want my forum to load faster”.
(An incredibly convoluted pricing model doesn’t help… I think there’s an XKCD for this, but I can’t find it.)

CloudFlare is not as configurable, but also offer quite a few other features as well - as well as the very approachable price tag of free. With a properly named bucket and a DNS rule (possibly supplemented by some caching rules), you get an instant improvement to loading performance, with no downsides whatsoever.
Perhaps not as big of a difference as a full CDN would make, but hey, it’s free performance.

3 个赞

@codinghorror @sam I am using a cloudflare CDN to access the S3 content (where discourse uploads all the files) and not directly.

s3 mandates to use bucket name like files.discourse.org to access its content through cname files

http://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html#VirtualHostingCustomURLs

Is there a way to allow periods in the S3 name (may be with a warning for https issue for direct s3 use). This is a showstopper in my case to use S3 with cloudflare and discourse.

Thanks

2 个赞

We worked around that here @gingerman :

5 个赞

@fearlessfrog Thanks…I ended up doing exactly the same. I don’t know how I missed your thread earlier. Just to highlight - this approach has a limitation in terms of cache invalidation. we have to do separately invalidations in cloudfront and cloudflare (after rebaking posts). It would be great to automate them with a script at the very least.

I hit up on the same blocks eventually in terms of optimised image urls and was planning to start a thread today…Nice to know that you have already helped the team towards a fix in the below given thread. Kudos for that.

2 个赞

我想为在 Discourse 中允许 S3 存储桶名称中包含点号(.)表示支持。AWS 现在已允许(甚至指导)在基于 CNAME 的 S3 内容访问中使用点号。只要 Discourse 中的 s3_cdn_url 设置被正确使用,据我所知,之前遇到的 HTTPS 问题已不再构成障碍,尤其是在使用 Cloudflare 等服务时(正如 @uppfinnarn 所指出的,基于 SNI 的 SSL 证书可以解决此问题)。

是否有可能在下一个版本中重新考虑这一问题?

感谢你们提供了如此优秀的产品!

既然我们已经支持将 S3_CDN_URL 设置为任意值,允许这样做能带来什么好处?

你好,Rafael,除非我漏掉了什么,虽然该参数允许 Discourse 从 CDN URL 拉取数据,但如果我们无法指定包含点的 S3 存储桶名称,那是否意味着实际上无法上传到这些存储桶?这迫使我们不得不采用一些权宜之计,例如在额外的非免费第三方服务(如 CloudFront)中设置备用 CNAME 记录(我们目前完全没有使用 CloudFront,并且更倾向于避免使用)。

(我猜归根结底,我只是不明白,为什么你会想要一概拒绝一个有效的 S3 存储桶名称!)

所以你们完全不用 HTTPS 吗?这就是上述问题的主要原因。

1 个赞

我不明白这里的权衡是什么。

如果我们允许带点的存储桶名称,就必须添加一个设置,以便在连接时禁用 SSL 验证;或者用户必须添加一个代理,在 Discourse 和 S3 端点之间建立 SSL 加密,但代理与 S3 端点之间仍然使用纯 HTTP。

做了所有这些,好处又是什么呢?

1 个赞

所以你们完全不使用 HTTPS……吗?这是上述情况的主要原因。

你好 Jeff,我们确实使用 HTTPS,通过 Cloudflare 实现。

如果我们允许包含点号的存储桶,就必须添加一个设置,让用户可以在连接时禁用 SSL 验证;或者,使用此功能的人必须添加一个代理,在 Discourse 和 S3 端点之间建立 SSL 连接,但代理与 S3 端点之间仍然会是纯 HTTP。

我相当确信 Cloudflare 会自动检测并处理这种情况,无需额外添加设置(我们之前在其他服务中也采用过这种方法)。

我完全理解您希望提醒用户避免使用您认为有问题的命名约定。但考虑到有效的 S3 存储桶名称可以包含点号(而且 AWS 文档也建议在多种场景下使用点号),您未来是否考虑过启用这一功能?

不,我们不会考虑这样做。

好的,谢谢。我想我们会打补丁或者找替代方案:slight_smile:

有什么替代方案?只需创建一个不带点号的新 S3 存储桶即可。我想不出任何理由要为 Discourse 的上传使用现有的存储桶。这样做甚至还能让财务核算更简单,因为里面不会有随机的非 Discourse 对象。

希望比 这个 更优化一些(但至少能工作)。

CloudFront CDN 是可选的。您可以使用任何其他 CDN,只要使用不含点号的存储桶名称,这样就不会遇到 HTTPS 相关的问题。

另外,读到这段对话时,我脑海中浮现的只有这张图片。

(警告:任何持有当前美国安全许可的人员不应在此公开此图片。)

2 个赞