Configure an S3 compatible object storage provider for uploads

Documentation Self-Hosting
, , ,
456

Has scaleway changed anything? It had worked for me until 48 hours ago but it no longer works.

1 Like
457

Is your bill paid? What is wrong? Do any images work? They are all gone or you just canā€™t add new ones?

1 Like
458

Can you add a little more support? For example, Tencent COS from China is more convenient for Chinese users, and Tencent Cloudā€™s COS has CDN nodes around the world, which is very fast to access

459

Have you tried it? Does it work?

460

I have used it in the test environment, but I can only back up to Tencent COS, and I hope that the discourse official can add Tencent COS support to facilitate developers from China

1 Like
461

I think that youā€™ll need to share what youā€™ve done and what errors or problems you had. Then someone might be able to suggest how to make it work. You might look at the other examples and see if you can use those to make it work.

I think itā€™s unlikely that theyā€™ll make changes in core to support it.

1 Like
462

Backblaze:

  • Iā€™ve setup a B2 bucket in my account
  • Added details to the env section of app.yml
  • Run ./launcher rebuild app

ā€¦but I donā€™t see any confirmation in my Discourse instance dashboard that these settings have taken effect. Where should I look?

1 Like
463

Hi,

I could successfully test and use Cloudflare R2 for both uploads and backups. Detailed post here Cloudflare R2: Navigating Setup and Handling Configuration Errors - #40 by kamaljain

Would request to kindly test at your end and review the ā€œWorks with Discourseā€ to Yes for Cloudflare R2. Infact its easier than S3 and more economical, free if <10gb of storage.

2 Likes
464

I too tested a setup on multisite and also think that itā€™s working.

Iā€™ve seen something that makes me wonder if you can get a list of backups. Iā€™ll try to check my setup.

1 Like
465

image
image1600Ɨ457 71.5 KB

I can see the backups listed. Havent tried the restore though.

1 Like
466

Hi bro, can I chat with you, Im have some problem when config with minio.
Thankyou.

467

iDrive Config

Just wanted to report that using iDriveā€™s E2 version of s3 storage works perfectly with these settings:

ENV settings in yaml file

Make sure your bucket is has at least Public & Read permissions in iDriveā€™s settings.

These are the settings I use for one bucket sharing uploads and backups.

DISCOURSE_USE_S3: true
DISCOURSE_S3_REGION: your-region
DISCOURSE_S3_ENDPOINT: https://your-idrive-endpoint.dev
DISCOURSE_S3_ACCESS_KEY_ID: your-access-key
DISCOURSE_S3_SECRET_ACCESS_KEY: your-secret-key
DISCOURSE_S3_CDN_URL: https://your-custom-cdn-url.com/uploads
DISCOURSE_S3_BUCKET: your-bucket-name/uploads
DISCOURSE_S3_BACKUP_BUCKET: your-bucket-name/backups
DISCOURSE_BACKUP_LOCATION: s3
DISCOURSE_S3_INSTALL_CORS_RULE: false

By default iDrive has Share everything in this region with every origin ā€¦ so it should work out of the box regardless of sources. Lock it down to your own domains if you want to.

I use a Cloudflare origin certificate to create a CNAME that points to the iDrive endpoint.

Sync Existing Attachments

Rebuild

Rebuild the app after changing these ENV settings.

Upload and Rebake

Then enter the container and run

rake uploads:migrate_to_s3

Watch it upload, then run

rake posts:rebake_uncooked_posts

Exit the container

Add to hooks section of yaml file

In hooks section

after_assets_precompile:
    - exec:
        cd: $home
        cmd:
          - sudo -E -u discourse bundle exec rake s3:upload_assets
          - sudo -E -u discourse bundle exec rake s3:expire_missing_assets

This ensures that js/css stuff gets uploaded to iDrive after your attachments do, otherwise youā€™ll get a bunch of console errors in the browser as theyā€™re all missing.

Rebuild

Rebuild again after assets command goes in.

Everything should be ok!

2 Likes
468

Perhaps Hetznerā€™s Object Storage service should be added to the table?

2 Likes
469

I managed to use Contabo for file uploads. Both old (migrated) files and new uploads worked just fine. In my case the bucket url was https://eu2.contabostorage.com/mybucket
The trick was to set EU2 as bucket name and https://contabostorage.com/mybucket as endpoint.
The only problem is that when loading the site it also requests other files like pluginā€™s js from the bucket and it doesnā€™t load being the files unavailable. Well, I understood this is because Iā€™ve set CDN url same as bucket. This isnā€™t very clear to me yet, can someone tell me how to set it up? From what I understood Iā€™ve to create a cname address that points to the main domain (discourse server) and proxy it with cloudflare, is this correct?

1 Like
470

You missed this bit:

1 Like
471

Iā€™m sure I did it, but maybe I made a mistake. In the end, I restored a backup, so Iā€™m going a bit by memoryā€¦ But then, I wonder, why is it discouraged to set a CDN as the Bucket? What problems can it cause? Will retry tomorrow.
Thanks

1 Like
472

I donā€™t think the problem you describe would be because you set the CDN url as the bucket url (that is, if uploads work, but assets donā€™t).

2 Likes
473

Will update you tomorrow, thanks

1 Like
474

And it worked :partying_face:
Step by step of what I did hoping to help others:

edited app.yml and added

  after_assets_precompile:
    - exec:        cd: $home
        cmd:          - sudo -E -u discourse bundle exec rake s3:upload_assets
          - sudo -E -u discourse bundle exec rake s3:expire_missing_assets

after hooks: (just after git clone pluginsā€¦)
and

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: EU
  DISCOURSE_S3_ENDPOINT: https://contabostorage.com/bucketname
  DISCOURSE_S3_ACCESS_KEY_ID: accesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: secretaccesskey
  DISCOURSE_S3_CDN_URL: https://eu2.contabostorage.com/randomlettersthatcontaboaddstothelinks:bucketname
  DISCOURSE_S3_BUCKET: eu2
  DISCOURSE_BACKUP_LOCATION: s3
  DISCOURSE_S3_INSTALL_CORS_RULE: false

after the email settings
then Iā€™ve set the same values in discourse menu and enabled S3 uploads.
Rebuilt the app with

cd /var/discourse
sudo ./launcher rebuild app

then

./launcher enter app
rake uploads:migrate_to_s3

This takes a bit, in my case a few hours (80k files, 100GB)

When finished

rake posts:rebake_uncooked_posts
4 Likes
475

Update: some users were unable to load the site (assets and files). Weā€™ve found that the contabostorage.com domain was present in some blocklists as a phishing site. The solution was simple:

setup a subdomain files.mydomain.com like described here Can I Setup a Custom Domain for My Object Storage? : Contabo

Updated DISCOURSE_S3_CDN_URL: https://files.mydomain.com in app.yml

Then

cd /var/discourse
sudo ./launcher rebuild app

and to complete the job

./launcher enter app
rake posts:rebake

Now all assets and uploads load from https://files.mydomain.com/pathtofile/file.ext

Done

2 Likes