Utilizing MinIO server for S3 compatible object storage
MinIO is an S3-compatible object storage server solution that is cloud-native by default but easy to deploy on on-prem servers or VPSes or cloud servers for your use as an alternative to Amazon AWS S3 or other systems, and is compatible with Discourse when configured right.
This section presupposes that the following are true in your environment:
You have a fully configured MinIO server instance
You have Domain Support enabled in the MinIO configuration
You have DNS configuration properly set up for MinIO so that bucket subdomains properly resolve to the MinIO server
The bucket discourse-data exists on the MinIO server and has a āpublicā policy set on it
The bucket discourse-backups exists on the MinIO server and is a private bucket for uploads to be put into (and not publicly reachable - the default policy for new buckets)
Your S3 CDN URL is a properly configured CDN pointing to the bucket and cache requests, as stated earlier in this document.
If you have all of the above requirements available, then you should be good to go.
Itās worked with both subdomain AND path style buckets for a while. However, the DNS configuration component is the most painful component - it needs special wildcard DNS rules to work, or a customized DNS server that is actively aware of all actual buckets or not. Iāve done that in bind9 with wildcard zones, but doing it in Cloudflare or others and the subdomain style bits barf hard.
IF you think the above is suitable to be in that section, Iāll be happy to. But itāll include that ācaveatsā section - and Iād appreciate any exits/reviews youād like me to make first for wording, etc. (And note I didnāt include a CDN URL 'cause I donāt have a CDN in my deployment (because $$$ is not something I can experiment with).
Users who run MinIO just for Discourse can also create the DNS entries for the 2 Discourse buckets manually too right?
The caveat in this topic should be enough. Configuring MinIO is completely out of scope for this forum, but given a working MinIO getting Discourse to use is a valid use case.
Correct, but as you said config is out of scope, so I simply will state that the bucket subdomain paths have to resolve (and will leave DNS config to ${ADMINS} of the instance)
Yep, and I"ve edited the wiki accordingly. However, MinIO doesnāt (to my knowledge) run a cloud service so I left the āService Nameā in the wiki TOC at the beginning blank, adjust accordingly if you wish.
I also tweaked it, Provider is Self-hosted but the MinIO section is still linked to. That should solve the fact they donāt have a cloud provided solution that I could find at the moment. (You can close this thread now if you wish since itās integrated into the wiki post)
Also discovered (thank you open source code and decent documentation for MinIO!) that CORS is already by default on all MinIO HTTP action verbs - so no need to install the CORS rules, theyāre already present. Also updated the self-hosted minio section as well, plus some grammar tweaks.
Thanks to @Falco for some help tracing the issue I had noticed during app build / rebuild time for this, and for the base guidance on CDN config with StackPath (because I wanted a fully functional test, and have StackPath CDN for something else anyways so using the CDN stack verified everything worked!)
I donāt get it how to set the force path style. When I set up version 2.6.8 with minio via the S3 settings it always prepends the bucket name to the s3 endpoint, where it should append it as path after the s3 endpoint.
It also looks from the config sample on github that the option s3_force_path_style has been removed. Am I missing something? Thank you.
Discourse uses DNS mode, not path mode, for S3. Itās been that way for a WHILE now, itās why I donāt specify anything about path mode in the how to OR the S3-storage how to page. You should also learn how to make new threads rather than necromancing an old thread up for something unrelated.
@teward Thank you for the feedback. Sorry, for necromancing, I am used to the GitHub issues way of writing I will start a new thread for how we can use discourse S3 in path mode.
You have Domain Support enabled in the MinIO configuration, for Domain driven bucket URLs. This is mandatory and not an option, there is no path-based support in Discourse for bucket paths.
A long time ago when I first was working with Discourse, they had that option, then it got removed, and I had to stop using MinIO as a backend. NOW, since thereās documentation in MinIO on how to get DNS mode working (i.e. bucket[.]server[.]com paths now like S3 does), it works when you properly configure MinIO. (Thank you to the staff/moderators/system for upping my trust level to edit the wiki itself now).