Configurar um provedor de armazenamento de objetos compatível com S3 para uploads

:information_source: Este tópico aborda como configurar alguns provedores comuns de armazenamento de objetos compatíveis com S3 (clones do S3). Consulte Set up file and image uploads to S3 para mais detalhes sobre a configuração do Amazon AWS S3, que é oficialmente suportado e usado internamente pelo Discourse para nossos serviços de hospedagem.

Provedor Nome do Serviço Funciona com o Discourse?
Amazon AWS S3 Sim
Digital Ocean Spaces Sim
Linode Object Storage Sim
Google Cloud Storage Sim
Scaleway Object Storage Sim
Vultr Object Storage Sim
BackBlaze Cloud Storage Sim*
Auto-hospedado MinIO Sim
Azure Blob Storage Flexify.IO Sim
Oracle Cloud Object Storage Não [1]
Wasabi Object Storage Talvez
Cloudflare R2 Sim
Contabo Object Storage Não

Se você conseguiu fazer um serviço diferente funcionar, adicione-o a esta wiki.

Configuração

Para armazenar ativos estáticos do Discourse em seu armazenamento de objetos, adicione esta configuração em seu app.yml na seção hooks:

  after_assets_precompile:
    - exec:
        cd: $home
        cmd:
          - sudo -E -u discourse bundle exec rake s3:upload_assets
          - sudo -E -u discourse bundle exec rake s3:expire_missing_assets

Ao usar armazenamento de objetos, você também precisa de uma CDN para servir o que é armazenado no bucket. Eu usei a CDN da StackPath em meus testes e, além de precisar definir Dynamic Caching By Header: Accept-Encoding em sua configuração, funciona bem.

DISCOURSE_CDN_URL é uma CDN que aponta para o nome do host do seu Discourse e faz cache das solicitações. Será usada principalmente para ativos puxáveis: CSS e outros ativos de temas.

DISCOURSE_S3_CDN_URL é uma CDN que aponta para o bucket do seu armazenamento de objetos e faz cache das solicitações. Será usada principalmente para ativos empurráveis: JS, imagens e uploads de usuários.

Recomendamos que sejam diferentes e que os administradores configurem ambas.

Não usar uma CDN (ou inserir a URL do bucket como a URL da CDN) provavelmente causará problemas e não é suportado.

Nos exemplos a seguir, https://falcoland-files-cdn.falco.dev é uma CDN configurada para servir os arquivos dentro do bucket. O nome do bucket foi definido como falcoland-files em meus exemplos.

Configurar essas variáveis de ambiente no seu app.yml é recomendado, pois é assim que a CDCK faz em sua infraestrutura, portanto, é bem testado. Além disso, a tarefa de upload de ativos ocorre após a compilação dos ativos, o que acontece em uma reconstrução. Se você deseja iniciar um Discourse que funcione corretamente com armazenamento de objetos desde o início, você precisa definir as variáveis de ambiente para que os ativos sejam enviados antes que o site seja iniciado.

Escolha seu provedor na lista abaixo e adicione essas configurações à seção env do seu arquivo app.yml, ajustando os valores conforme necessário:

AWS S3

O que oficialmente suportamos e usamos internamente. Sua oferta de CDN, Cloudfront, também funciona para servir os arquivos do bucket. Consulte Set up file and image uploads to S3 para saber como configurar as permissões corretamente.

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: us-west-1
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backups
  DISCOURSE_BACKUP_LOCATION: s3

Digital Ocean Spaces

A oferta da DO é boa e funciona imediatamente. É seguro ativar a opção “Restrict File Listing” (Restringir listagem de arquivos). O único problema é que sua oferta de CDN está terivelmente quebrada, então você precisa usar uma CDN diferente para os arquivos. Além disso, você não deve instalar a regra CORS, pois ela a reinstala a cada reconstrução.

Exemplo de configuração:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: whatever
  DISCOURSE_S3_ENDPOINT: https://nyc3.digitaloceanspaces.com
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backups
  DISCOURSE_BACKUP_LOCATION: s3
  DISCOURSE_S3_INSTALL_CORS_RULE: false 

Linode Object Storage

Um parâmetro de configuração extra, HTTP_CONTINUE_TIMEOUT, é necessário para o Linode.

Exemplo de configuração:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: us-east-1
  DISCOURSE_S3_HTTP_CONTINUE_TIMEOUT: 0
  DISCOURSE_S3_ENDPOINT: https://us-east-1.linodeobjects.com
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backup
  DISCOURSE_BACKUP_LOCATION: s3

Google Cloud Platform Storage

A listagem de arquivos está quebrada, então você precisa de uma variável de ambiente extra para ignorá-la para que os ativos funcionem. Além disso, ignore o CORS e configure-o manualmente.

:warning: Como você não pode listar arquivos, não poderá listar backups e os backups automáticos falharão, portanto, não recomendamos o uso para backups. No entanto, alguns sugerem que, se você alterar a função de Storage Legacy Object Owner para Storage Legacy Bucket Owner, os backups funcionam corretamente. Consulte este tópico para discussões específicas do Google Cloud.

Existe um plugin de terceiros para melhorar a integração em Discourse GCS Helper.

Exemplo de configuração:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: us-east1
  DISCOURSE_S3_INSTALL_CORS_RULE: false
  FORCE_S3_UPLOADS: 1
  DISCOURSE_S3_ENDPOINT: https://storage.googleapis.com
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  #DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backup
  #DISCOURSE_BACKUP_LOCATION: s3

Scaleway Object Storage

A oferta da Scaleway também é muito boa e tudo funciona bem na maioria das vezes.

:warning: Os uploads multipart da Scaleway suportam apenas um máximo de 1.000 partes. Isso não corresponde ao Amazon S3, que suporta um máximo de 10.000 partes. Para instâncias maiores, isso fará com que o Discourse falhe nos backups e o upload incompleto pode precisar ser excluído manualmente antes de novas tentativas. Para instâncias pequenas, isso não é um problema. A Scaleway parece bastante aberta a feedback, então, se você quiser que esse limite seja alterado, entre em contato com eles.

Observe que, para o parâmetro DISCOURSE_S3_ENDPOINT, o Discourse usa o endpoint de toda a região: https://s3.{region}.scw.cloud. O “Bucket endpoint” encontrado no seu painel da Scaleway vem no formato https://{bucketName}.s3.{region}.scw.cloud. Omita o subdomínio do nome do bucket para evitar erros de conexão.

Exemplo de configuração:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: fr-par
  DISCOURSE_S3_ENDPOINT: https://s3.fr-par.scw.cloud
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backups
  DISCOURSE_BACKUP_LOCATION: s3

Vultr Object Storage

Um parâmetro de configuração extra, HTTP_CONTINUE_TIMEOUT, é necessário para o Vultr.

Exemplo de configuração:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: whatever
  DISCOURSE_S3_HTTP_CONTINUE_TIMEOUT: 0
  DISCOURSE_S3_ENDPOINT: https://ewr1.vultrobjects.com
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backup
  DISCOURSE_BACKUP_LOCATION: s3

Backblaze B2 Cloud Storage

Você precisa ignorar o CORS e configurá-lo manualmente.

relatos de que a função “limpar uploads órfãos” não funciona corretamente com a BackBlaze. Você deve alterar as regras de ciclo de vida do seu bucket para que a limpeza de órfãos funcione.

Exemplo de configuração:

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: "us-west-002"
  DISCOURSE_S3_INSTALL_CORS_RULE: false
  DISCOURSE_S3_CONFIGURE_TOMBSTONE_POLICY: false
  DISCOURSE_S3_ENDPOINT: https://s3.us-west-002.backblazeb2.com
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backup
  DISCOURSE_BACKUP_LOCATION: s3

Nota: Durante a migração inicial para o B2, você pode atingir o limite de 2.500 transações gratuitas diárias da classe C. Você precisará adicionar um método de pagamento para remover os limites.

MinIO Storage Server

Existem algumas ressalvas e requisitos que você precisa garantir que sejam atendidos antes de poder usar o servidor de armazenamento MinIO como alternativa ao S3:

  1. Você tem uma instância de servidor MinIO totalmente configurada
  2. Você tem o suporte a domínio ativado na configuração do MinIO, para URLs de buckets baseados em domínio. Esta é uma configuração obrigatória para MinIO e Discourse, pois o MinIO ainda suporta os estilos “path” legados do S3, que não são mais suportados no Discourse.
  3. Você tem a configuração de DNS corretamente definida para o MinIO, para que os subdomínios do bucket resolvam corretamente para o servidor MinIO e o servidor MinIO esteja configurado com um domínio base (neste caso, minio.example.com)
  4. O bucket discourse-data existe no servidor MinIO e tem uma política "pública

  1. Oracle Cloud não oferece suporte a acesso ao estilo virtual-host para buckets e não funcionará ↩︎

69 curtidas
Defining DISCOURSE_S3_CDN_URL links to assets in S3 CDN URL
Backblaze S3 issue: duplicated uploads after delete
Migrate from AWS to Digital Ocean with 2 containers, spaces and 2 CDNs
Using Scaleway s3-compatible object storage
Extend S3 configuration for other s3 API compatible cloud storage solutions
Setting up backup and image uploads to Backblaze B2
What are the right settings to use S3 bucket (with non-Amazon URL)?
Upload assets to S3 after in-browser upgrade
Using multiple containers - what needs to be shared?
Virus scanning of uploaded files
Imgur images broken
Admin role conflates server admin and board admin
Error in rebuilding using minio as object store
Use WebTorrent to load media objects
Issues with changing File/Image upload location to S3 Server from local storage
Hosting Optimization with Digital Ocean
Hosting Optimization with Digital Ocean
Theme modifiers: A brief introduction
Configure automatic backups for Discourse
Problem with Backblaze for backup- Failed to list backups from S3: Signature validation failed
Move from BackBlaze B2 to Digital Ocean Spaces
Which free storage for many images? also to be used for thumbnails etc
Migrate from AWS to Digital Ocean with 2 containers, spaces and 2 CDNs
Restore Failure - S3 (compatible) backup
Restore Failure - S3 (compatible) backup
Digitalocean block storage VS amazon S3
Digitalocean block storage VS amazon S3
Admin upgrade page doesn't load with a CDN
Install Discourse for Production Environment on Windows Server
Running Discourse on Azure Web Sites vs. Azure VM?
How to turn off S3 storage?
Access Denied error message when trying to upload images
What are the right settings to use S3 bucket (with non-Amazon URL)?
REQ: Support S3 backup to a service like Backblaze
REQ: Support S3 backup to a service like Backblaze
Using Scaleway s3-compatible object storage
Overwrite meta og:image image source to use externally public loaded images on topics?
How to store uploads with multiple web_only servers?
Can not edit topics with picture with S3 backend
Finding UI generated backup and restoring site
Looking for doc to connect discourse with digital ocean spaces
Looking for doc to connect discourse with digital ocean spaces
Looking for doc to connect discourse with digital ocean spaces
403 Error with digital ocean cdn
Link to headers (anchor links)
NoMethodError downcase s3_bucket_name absolute_base_url
What should I enter in the S3 CDN settings if I don't have a CDN?
Backing up files in Object Storage
Minio: A header you provided implies S3 functionality that is not implemented
Configure automatic backups for Discourse
S3 OVH Object Storage
File directory system
Unable to backup or navigate to backups
Uploads to AWS S3 and Configured CDN
Migration to a Self-Hosted solution from Kubernetes
How to develop discourse in a team?
Images didn't load after configuring S3 uploads
Need Azure blob storage
Use Google Cloud Storage Instead of S3
Move discourse uploads to free space on server
Strange behavior with Digital Ocean Spaces/S3 and lots of duplicated CORS rules for `example.com`
Can't upload PDF to S3
"canned acl" error when uploading images
AWS S3 Object Ownership
What’s your server configurations?
How to debug S3 uploads
Problems with Patreon Login, Force HTTPS, and S3 CDN (three) Issues
Cron task to sync local backups to DigitalOcean Spaces
Cron task to sync local backups to DigitalOcean Spaces
Problems with Patreon Login, Force HTTPS, and S3 CDN (three) Issues
Help restoring - system hung at midnight
Help restoring - system hung at midnight
Image upload error: The bucket does not allow ACL's
Make s3_region a string value
How can I set s3_force_path to true?
S3 assets folder, is a cleanup needed?
BunnyCDN storage for automatic backups
BunnyCDN storage for automatic backups
Move Uploads and Backups to DigitalOcean Block Storage
Required local storage for URL referenced images vs. uploaded images?
Colored loading dots because assets not uploaded
Is DO Spaces CDN still broken? Any workarounds?
Is DO Spaces CDN still broken? Any workarounds?
Is DO Spaces CDN still broken? Any workarounds?
Basic How-To for Using MinIO storage server run by you for your Discourse Instance
Minio instead of S3?
Configure automatic backups for Discourse
S3 image bandwidth costs are getting annoying
S3 image bandwidth costs are getting annoying
Support for ImgBB image upload service
Install Discourse on Amazon Web Services (AWS)
Account creation not working with discourse_encrypt
Cannot load any images “Uncaught (in promise) no-response: no-response”
Tips on Google Cloud S3
Enable a CDN for your Discourse
Invalid URL for markdown-it-bundle
Not using volumes (or automatically using S3) when hosting with Docker
Configuring automatic backups
I have a problem with my proxy status
Inline PDF Previews
My images are not working
Getting error 422 invalid argument when configure S3 upload with GCS
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
Strange behavior with Digital Ocean Spaces/S3 and lots of duplicated CORS rules for `example.com`
My install broke after updating, how can I fix it?
Uploads Subdomain For Cloudflare
Hetzner deleted my account, my server and I was left with nothing. What to do?
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
Understanding Uploads, Images, and Attachments
S3, Assets & CDN Error
Capacity planning / Resource requirements
Migrate assets to s3 rake tasks
Best option for DB and file Storage
S3 error, when updating to 2.9.0.beta1
Forum rendered unusable
Forum rendered unusable
Configure automatic backups for Discourse
Migrate Discourse broke S3 images
Configure automatic backups for Discourse
Cloud storage and cdn
Migrating uploads from S3 to local
Another discourse offline "bootstrap failed with exit code 5"
Another discourse offline "bootstrap failed with exit code 5"
AI Plugin Build Error - 'bundle exec rake db:migrate' failed
Your Docker installation is not working correctly - no space left on device - safe to use ./launcher cleanup?
Uncaught SyntaxError: forum spins nonstop and never loads
Why is my upload limit 100mb?It's set to 500mb
Trouble with Google Bucket for backup
Did I overwrite my site settings with a cross-instance backup/restore?
Need tips on making my forum faster
What should be the server requirements
SSL_connect returned=1 errno=0 peeraddr=162.243.189.2:443 state=error: certificate verify failed (Hostname mismatch)
Set up BackBlaze S3 with BunnyCDN
Why you should use Discourse internally for your company/team instead of Slack (4 years use case)
Make s3_region a string value
Imgur upload built into the post buttons
Decoupled Discourse Application - Managed Redis, Managed Postgres, and DIgital Ocean Volume with Discourse
S3 and Cloudfront - Setup Not Working
Cannot upload images in mobile discourse
Cloudflare R2: Navigating Setup and Handling Configuration Errors
Cloudflare R2: Navigating Setup and Handling Configuration Errors
Cloudflare R2: Navigating Setup and Handling Configuration Errors
S3 assets broken after moving forum
Backup request via FTP
Error when doing "bundle exec rake s3:upload_assets" - Could not locate Gemfile or .bundle/ directory
Separate S3 access keys for backups and uploads?
Topic List Previews (legacy)
Disk usage spike during backup, Discourse crashed hard :-(
Make s3_region a string value
Favicon setup: "To work correctly over a CDN it must be a png"
Change image folder to symlinked folder
Migrating uploads from S3 to local
S3 Upload Confusion: Dashboard vs. app.yml
Old installation failing upgrade
Where are Images and uploads stored?
Rebuild issue: [Cannot set dual-stack in combination with a custom endpoint.]
Problem updating data in the separate database
Resizing images off server
Unable to setup S3 bucket
Discourse Stuck on Loading (Logs)
Discourse Stuck on Loading (Logs)
Unable to setup S3 bucket
How to add AWS Cloudfront as a Discourse CDN
Unable to setup S3 bucket
Configured s3, but i want asset to serve locally
Configured s3, but i want asset to serve locally
Set up BackBlaze S3 with BunnyCDN
Discourse s3 backup folder
SSO broken after rebuild with stable v3.3.3
How to Configure Cloudflare R2 for your Discourse Community
如何批量修改文件链接地址
Can't migrate uploads to S3
Backup discourse from the command line
Stuck and lost updating forum, problems with PG migration
Can't rebuild due to AWS SDK gem bump and new AWS Data Integrity Protections
Can't rebuild due to AWS SDK gem bump and new AWS Data Integrity Protections
S3 (not AWS) backups stopped working, presumably since an update
Troubleshooting S3 Uploads: Site hangs after rebuild, JS assets fail to load with net::ERR_... on both R2 and GCS
Inquiry About Discourse’s Support for Amazon S3 Storage
Backup discourse from the command line
Inquiry About Discourse’s Support for Amazon S3 Storage
Inquiry About Discourse’s Support for Amazon S3 Storage
Cloudflare R2 Storage Issues
Hetzner S3 support
Can the /var/discourse and /var/lib/docker folders be on different volumes?
Setting up backup and image uploads to Backblaze B2
Issues with AWS CDN and S3
Digital Ocean Spaces (S3) "unable to sign request without credentials set"
Issues with AWS CDN and S3
Digital Ocean Spaces (S3) "unable to sign request without credentials set"
Digital Ocean Spaces (S3) "unable to sign request without credentials set"
Digital Ocean Spaces (S3) "unable to sign request without credentials set"
Digital Ocean Spaces (S3) "unable to sign request without credentials set"
为啥我的七牛云s3附件上传成功后,论坛中无法加载出来?
Install Discourse on a residential internet with Cloudflare Tunnel
Custom emoji don't use CDN for S3 stored assets in a few pages
S3 and Cloudfront - Setup Not Working
Secure Uploads
Make s3_region a string value
Backup Files Not Showing in Discourse Backend After Uploading to Cloudflare R2
Images after a restore don't have S3 bucket URL
Can't upload to S3 (Cloudflare R2)
Daily Summary (9pm UTC)
Custom Emojis loaded from S3/R2 bypass CDN routing
An AWS S3 Integration with Discourse Tutorial would be nice
Is Scaleway a good option for object storage?
Blackblaze&Cloudflare: image uploading works, but not showing in post
SMF2 Conversion and Rake to S3 Help
Backing up your forum
What causes rake uploads:fix_relative_upload_links
Running 2 hosts behind haproxy fails with random 404s
Site Blank After Rebuild
Rebuild goes into a loop
Custom Emojis loaded from S3/R2 bypass CDN routing
Backblaze S3 issue: duplicated uploads after delete
Migrate_to_S3 Fails on Rebake
Downloads coming from S3 even with DISCOURSE_S3_CDN_URL set
Errors trying to use custom S3 storage
Moving from one S3 bucket to another
S3 image bandwidth costs are getting annoying
Basic How-To for Using MinIO storage server run by you for your Discourse Instance
SSL error - can't upload images
Digital Ocean Spaces don’t implement the AWS S3 API for the CORS rule
Extend S3 configuration for other S3 API compatible services
How to separate uploaded files to another server
Storing Images in the cloud
S3 and Cloudfront - Setup Not Working
Migrate_to_S3 Fails on Rebake
Can not access backup page and related error when restoring using GCP Object Storage
Cannot rebake after setting up CDN
Rebuild failure - skipping "after_assets_precompile" section of app.yml
High Availability 3 Server setup
Enable hidden setting to include S3 uploads in the backups
Run Discourse as docker swarm service
Azure Blob Storage Plugin
How might we better structure #howto?
Migrating uploaded files from DO to S3
Discourse as a closed wiki