Definindo links DISCOURSE_S3_CDN_URL para ativos na URL do CDN S3

Suporte
1

I had this problem before and decided that I was crazy, confused, or the database on the site was suspect, but this is on a brand new site. Also, I was on Digital Ocean spaces, so I thought it might be a problem somehow.

I’m trying again to figure out how to keep images on AWS S3 like I think the Big Boys do.

Here’s what I have in the env section:

  DISCOURSE_S3_ACCESS_KEY_ID: 'key'
  DISCOURSE_S3_SECRET_ACCESS_KEY: 'lock'
  DISCOURSE_BACKUP_LOCATION: 's3'
  DISCOURSE_S3_BUCKET: 'lc-xyz'
  DISCOURSE_S3_BUCKET_NAME: 'lc-xyz'
  DISCOURSE_S3_BACKUP_BUCKET: 'lc-xyz/backups'
  DISCOURSE_S3_UPLOAD_BUCKET: 'lc-xyz/uploads'
  DISCOURSE_S3_CDN_URL: 'https://lc-rbx.s3.amazonaws.com'

When I include the s3 cdn url the site breaks because all of the links to the assets are on S3 like https://lc-xyz.s3.amazonaws.com/assets/plugin-third-party-01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b.br.js.

Because if you define s3 cdn url Discourse looks for the assets on the s3_cdn_url.

I did a rebuild, but the assets are still missing. I can do a

rake s3:upload_assets

Is there an after_bundling_assets stanza that I could add that to? (I know about after_db_migrate and after_bundle_exec, but don’t know if those would work.)

Do assets get produced some other time? (It would seem like when themes get modified assets would change.)

If I also had a “push CDN like normal”, would that keep this from happening?

Is there some best practice that I’m missing?

2

You can use after_assets_precompile as the hook for that rake task.

3

Aha! Thanks very much for that. Where does the list of those things live?

I’m still confused why this is necessary, especially since you seemed confused before. (Or maybe I should be happy that I can push all this stuff to S3, now that I know how to do it–And I think that it was you who said that one could use CloudFlare’s free CDN to front an AWS bucket.)

4

I’m trying this again. I’ve got uploads going to s3. That works.

I configured cloudflare to CDN the site. If I enter https://lc-XXX.literatehosting.com/uploads in the s3_cdn_url site setting, new uploads go to the S3 bucket and the image is linked to the CDN url and it works.

BUT if I try to set the s3_cdn_url with an ENV in app.yml (or edit config/discourse.conf by hand and sv restart unicorn, all of the assets are loaded from S3 (where they aren’t). That might be OK, but if I try to rake s3:upload_assets, it complains that S3 isn’t configured.

5

I happened across this while figuring out some clashing settings names.

I think (and hope, because it means I understand this and didn’t write absolute nonsense in the thread linked above) that if you remove these two lines:

DISCOURSE_S3_BUCKET: 'lc-xyz'
DISCOURSE_S3_BUCKET_NAME: 'lc-xyz'

Then you can set DISCOURSE_S3_CDN_URL, and it will only be used for uploads, not assets.

6

I think that you do understand, and, thanks to you, I’m at least closer to understanding myself! Thanks very much!

7

Running into this and still very confused after reading this thread…

I’m trying to server s3 uploads (not compiled assets) from CDN (Cloudfront).

If I configure “s3 cdn url” via the settings screen, it works as expected (well…except for System upload not using s3 cdn url)

However, if I configure via DISCOURSE_S3_CDN_URL and rebuild, the frontend is broken because its trying to load compiled assets from my s3 cdn url.

Seems DISCOURSE_S3_CDN_URL / s3_cdn_url should only affect uploads, and DISCOURSE_CDN_URL should only affect assets.

8

That is my experience too. I ended up making a plugin to set the S3_CDN_URL.

9

@pfaffman seems like this should be filed as a bug then, yes?

10

Perhaps. It’s not a feature likely to be used by normal self hosters,so it’s going to be a low priority. Also, I think that there is soon to be a change in how global settings and shadowed by global work, so it’ll likely get worked out then.

11

Parece que o mesmo problema me rejeitou aqui.
@pfaffman, preciso da sua ajuda.
Você usa o CloudFront para alcançar a URL de CDN de domínio personalizado ou apenas usa o prefixo público dos objetos do bucket na URL da CDN?

12

Acabei de me deparar com isso. @pfaffman, você conseguiu resolver isso?

Consegui fazer os uploads funcionarem configurando manualmente o s3_cdn_url nas configurações do site, mas idealmente preciso poder definir a variável de ambiente globalmente ao fazer o deploy. Quando faço isso, tenho o mesmo problema: o Discourse tenta buscar os assets no s3_cdn_url, o que parece ser um bug para mim em discourse/lib/content_security_policy/default.rb at main · discourse/discourse · GitHub

13

Acho que acabei de definir o valor no banco de dados.

Minha suposição é que a solução seja encontrar a tarefa rake que enviará os ativos para o S3.

14

Estava me perguntando se alguém da @team poderia confirmar se esse é o comportamento esperado, de que não é possível usar uma CDN S3 (configurada globalmente) para uploads sem que o Discourse também procure os ativos lá. Parece um comportamento inesperado, a menos que eu esteja entendendo algo errado.

15

Sim, esse é o comportamento.

Configurar o CDN do S3 fará com que ele seja usado para tudo o que seja um arquivo estático, sejam uploads ou ativos JS.

16

Aqui estão algumas informações sobre isso; eu lidei com isso no mês passado.

Corrigi o problema configurando ambas as variáveis (DISCOURSE_S3_CDN_URL e DISCOURSE_CDN_URL) e criando duas distribuições do CloudFront: uma para os uploads, com origem no bucket S3, e outra para os ativos, com origem no servidor.

Aqui está o código que usamos para isso:

Aqui está nosso arquivo app.yml (que chamamos de web.yml), onde substituímos as variáveis no momento da construção: infra/modules/services/discourse/web.yml at master · debtcollective/infra · GitHub

17

Incrível. Muito obrigado. Parece não haver muita documentação sobre isso, e as configurações sugerem que você pode usar o S3 apenas para uploads, o que, creio eu, é o que está confundindo a todos nós.

18

Está na minha lista escrever um howto sobre isso. Espero fazer isso até o final de semana.

19

Sim, assim que você entende que precisa de duas distribuições do CloudFront, faz mais sentido. Além disso, lembre-se de fazer upload dos ativos para o S3 após cada reconstrução. Há um problema nas atualizações a partir do Docker Manager, onde é necessário executar bundle exec rake s3:upload_assets manualmente dentro do contêiner Docker. Se você fizer uma reconstrução em vez disso, deve funcionar se adicionar estas linhas ao seu app.yml

hooks:
  after_assets_precompile:
    - exec:
        cd: $home
        cmd:
          - 'bundle exec rake s3:upload_assets'
20

Essa tarefa está reclamando que eu não configurei o S3. Eu reduzi o problema a este método em global_setting.rb:

def self.use_s3?
    (@use_s3 ||=
      begin
        s3_bucket &&
        s3_region && (
          s3_use_iam_profile || (s3_access_key_id && s3_secret_access_key)
        ) ? :true : :false
      end) == :true
  end

Onde GlobalSetting.s3_bucket deve ser definido? Parece-me que precisamos definir as variáveis de ambiente DISCOURSE_S3_UPLOAD_BUCKET e DISCOURSE_S3_BUCKET. Qual é a diferença entre as duas?