# Extend S3 configuration for other S3 API compatible services
**URL:** https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371
**Category:** Feature
**Created:** [6월 21, 2018, 7:10오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371 "2018-06-21T07:10:42Z")
**Posts on this page:** 20
**Page:** 2
### Author:  [@arrowcircle](https://meta.discourse.org/u/arrowcircle)
#### Post date: [6월 29, 2018, 12:24오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/21 "2018-06-29T12:24:20Z")
Why not add `force_path_style` as a separate option?
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [6월 29, 2018, 12:26오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/22 "2018-06-29T12:26:58Z")
> [@arrowcircle](#):
>
> Why not add `force_path_style` as a separate option?
Yes, I already did that!
You can view the PR [here.](https://github.com/discourse/discourse/pull/6045)
---
### Author:  [@arrowcircle](https://meta.discourse.org/u/arrowcircle)
#### Post date: [6월 29, 2018, 12:28오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/23 "2018-06-29T12:28:52Z")
Thank You!
Any plans on adding nginx template to docker\_discourse repo?
I think its common thing for minio/s3/spaces storage - caching is absolutely needed.
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [6월 29, 2018, 3:38오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/24 "2018-06-29T15:38:04Z")
No problem, I will need to learn more about nginx caching before adding anything but I will look into it.
> [@Falco](#):
>
> Maybe after we merge the PR we can work out an optional template with nginx caching and another one using CDNs (Cloudflare should fit here, since it will be caching only statics).
This 👍
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [6월 29, 2018, 3:42오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/25 "2018-06-29T15:42:47Z")
Yes, and any form of nginx caching is a PR to [another repo](https://github.com/discourse/discourse_docker), so let’s get this done first.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [6월 29, 2018, 3:58오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/26 "2018-06-29T15:58:25Z")
> [@rishabh](#):
>
> Sure, I’ve added that and the PR now works great with S3, DigitalOcean Spaces and Minio 😀
Shouldn’t this setting be a checkbox (boolean)?
So this:
```plaintext
s3_force_path_style:
default: 'false'
regex: '^(true|false)$'
```
becomes:
```plaintext
s3_force_path_style:
default: false
```
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [6월 29, 2018, 5:43오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/27 "2018-06-29T17:43:55Z")
> [@Falco](#):
>
> Shouldn’t this setting be a checkbox (boolean)?
Yes, absolutely.
I’ve updated and tested the PR 👍
* * *
Whenever this is merged, may I write guides on Meta for setting up Spaces/Minio?
(Similar to [Setting up file and image uploads to S3](https://meta.discourse.org/t/setting-up-file-and-image-uploads-to-s3/7229))
---
### Author:  [@Alankrit\_Choudh](https://meta.discourse.org/u/Alankrit_Choudh)
#### Post date: [6월 30, 2018, 5:44오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/28 "2018-06-30T05:44:50Z")
Will there be a migrate script to and fro discourse—spaces?
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [6월 30, 2018, 6:14오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/29 "2018-06-30T06:14:36Z")
I just realised `enable_s3_uploads` and `enable_s3_backups` go through very different code paths. I initially assumed that all s3 objects are uploaded the same way. (Sorry, I’m super new here)
The PR currently breaks `enable_s3_uploads` because the final url is hardcoded for AWS uploads here:
In site\_setting.rb:
```plaintext
def self.absolute_base_url
bucket = SiteSetting.enable_s3_uploads ? Discourse.store.s3_bucket_name : GlobalSetting.s3_bucket_name
# cf. http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
if SiteSetting.Upload.s3_region == "us-east-1"
"//#{bucket}.s3.amazonaws.com"
elsif SiteSetting.Upload.s3_region == 'cn-north-1'
"//#{bucket}.s3.cn-north-1.amazonaws.com.cn"
else
"//#{bucket}.s3-#{SiteSetting.Upload.s3_region}.amazonaws.com"
end
end
```
_Note: I’m working on it, do not merge._
---
### Author:  [@ckshen](https://meta.discourse.org/u/ckshen)
#### Post date: [6월 30, 2018, 6:18오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/30 "2018-06-30T06:18:36Z")
Wouldn’t the existing rake task work? The new code just point to a new storage location.
---
### Author:  [@itsbhanusharma](https://meta.discourse.org/u/itsbhanusharma)
#### Post date: [6월 30, 2018, 8:48오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/31 "2018-06-30T08:48:37Z")
No because those tasks in their current state are made to work specifically with AWS with hardcoded parameters!
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [6월 30, 2018, 9:28오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/32 "2018-06-30T09:28:49Z")
Yeah, I’ll have to be very careful when changing the `absolute_base_url` return values because it’s being used all over the codebase (almost 40 usages) and I don’t want the change to break anything else 😃
---
### Author:  [@itsbhanusharma](https://meta.discourse.org/u/itsbhanusharma)
#### Post date: [6월 30, 2018, 9:31오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/33 "2018-06-30T09:31:13Z")
Or you can perhaps copy that task and create a separate task to handle DO?
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [7월 3, 2018, 11:58오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/34 "2018-07-03T11:58:20Z")
I’ve kept the older AWS region logic as it is (didn’t want to refactor unless there was a need to) and just added support for a custom endpoint if the default endpoint isn’t used:
```plaintext
def self.absolute_base_url
url_basename = SiteSetting.s3_endpoint.split('/')[-1]
bucket = SiteSetting.enable_s3_uploads ? Discourse.store.s3_bucket_name : GlobalSetting.s3_bucket_name
# cf. http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region
if SiteSetting.s3_endpoint == "https://s3.amazonaws.com"
if SiteSetting.Upload.s3_region == "us-east-1"
"//#{bucket}.s3.amazonaws.com"
elsif SiteSetting.Upload.s3_region == 'cn-north-1'
"//#{bucket}.s3.cn-north-1.amazonaws.com.cn"
else
"//#{bucket}.s3-#{SiteSetting.Upload.s3_region}.amazonaws.com"
end
else
"//#{bucket}.#{url_basename}"
end
end
```
* * *
**Update:**
I’m happy to report that uploads and backups are working 100% as expected for DigitalOcean Spaces with the PR. 🎉 (image uploads, bucket creation, backup uploads and deletion 👍) The code will work for platforms that are fully S3-compatible.
Note: _uploads and backups work well for Minio too, it’s just that the images don’t show up on Discourse because Minio routes their object urls differently. It’s a minio configuration issue and I’m working on it but does the code look okay till now?_ @Falco
---
### Author:  [@Alankrit\_Choudh](https://meta.discourse.org/u/Alankrit_Choudh)
#### Post date: [7월 4, 2018, 3:31오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/35 "2018-07-04T15:31:31Z")
Hey, is the PR in effect so we can use digital ocean spaces?
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [7월 4, 2018, 3:41오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/36 "2018-07-04T15:41:04Z")
Not yet.
We have hundreds of people using S3 currently, so we must be very careful here so we don’t break current setups.
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [7월 5, 2018, 11:29오전 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/37 "2018-07-05T11:29:21Z")
**Update:** Minio now works perfectly for uploads/backups 🎉
* * *
I can refactor the `absolute_base_url` method in various ways but I’ll wait till I get a review from the team (because we have to be careful with code/logic changes).
> [@Falco](#):
>
> We have hundreds of people using S3 currently, so we must be very careful here so we don’t break current setups.
Let me know if I can do anything to help 👍
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [7월 12, 2018, 7:48오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/38 "2018-07-12T19:48:06Z")
the PR is merged.
@rishabh can you confirm if it works with uploads?
I have a question. If we enable this feature, what would happen to previous uploads? Those that are on our own disk. Do they automatically transfer? On the other hand, if some day we might want to move back the uploads to our own server and turn off this feature, How can we migrate data back?
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [7월 16, 2018, 12:46오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/39 "2018-07-16T12:46:48Z")
Yes, it works with uploads! 👍
No, previous uploads would be not be affected. They don’t transfer automatically and only the future uploads will be uploaded to the new provider when you switch. I will check if it’s possible/feasible to migrate from one method to another.
---
### Author:  [@Alankrit\_Choudh](https://meta.discourse.org/u/Alankrit_Choudh)
#### Post date: [7월 16, 2018, 12:50오후 UTC](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371/40 "2018-07-16T12:50:40Z")
awesome, so how do we setup digitalocean spaces any tutorials out there?
[Previous page](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371.md?page=1)
[Next page](https://meta.discourse.org/t/extend-s3-configuration-for-other-s3-api-compatible-services/90371.md?page=3)