S3 competitor integration

Contribute Feature

AI-generated summary

The discussion centers on exploring S3 alternatives due to its complex pricing, especially for large Discourse instances heavy on media. Frank proposes investigating Rackspace/OpenStack or Google Cloud, noting Google’s pricing is straightforward and suggesting a feasibility study. He also references @zogstrip’s prior work on normalizing image and backup storage. zogstrip responds that adding a new file storage backend shouldn’t be overly difficult, as it only requires implementing a basic interface — though testing and ensuring compatibility (especially with ATIP) remains challenging. Frank acknowledges the significant time investment required, referencing Evil Trout’s testing tools, but argues it’s a necessary evil given past regressions. He also hints that integrating Google Cloud or S3-like bucket types may require additional functions, though the core interface appears clean and simple. The thread reflects cautious optimism about expanding storage options while acknowledging the engineering overhead involved. Basic interface reference

1

This is a continuation of a discussion to find an S3 competitor.

The main reason is that S3 has bizarre pricing structures and IMO for a large Discourse instance including those based on photo and video it could hit them for a lot of $$$$

Proposal:

Feasibility investigation on Rackspace/Openstack or Google Cloud and the extension/continuation of @zogstrip 's normalization work on image bucket and backup storage. So far the pricing seems very straight forward for Google Cloud hosting. But can it be done? or is it a high threshold pain point that triggers fight or flight instincts? I guess I will judge that by how fast this post gets deleted or closed and killed with fire :smile:

Disclaimer:

I could do a PR but I am dynamic/procedural candy coder, anything I provide via a PR will be murdered instantly which would wreck my fragile psychology. So someone with thicker skin would need to do this.

2

Adding a new file storage shouldn’t be that hard since they only need to implement a basic interface. The hardest part will be adding proper test coverage. Testing I/O truly is a PITA :frowning:

2 „Gefällt mir“
3

I have seem some of your testing ‘stuff’ (evil trout’s actually). And it looks like a HUUUUUUUUGGGE time sink (ice pick to the eyeballs).

but…I believe with the regression that occurred in the past, it’s a necessary evil, right?

:dizzy:

Hmmm…that basic interface looks really simple and clean. I think it would need an additional function of

def bucketType (S3/Google) (or something that carries some routing logic through to the other functions)

Interesting…