Definire i link DISCOURSE_S3_CDN_URL per le risorse nell'URL CDN S3

Supporto
1

I had this problem before and decided that I was crazy, confused, or the database on the site was suspect, but this is on a brand new site. Also, I was on Digital Ocean spaces, so I thought it might be a problem somehow.

I’m trying again to figure out how to keep images on AWS S3 like I think the Big Boys do.

Here’s what I have in the env section:

  DISCOURSE_S3_ACCESS_KEY_ID: 'key'
  DISCOURSE_S3_SECRET_ACCESS_KEY: 'lock'
  DISCOURSE_BACKUP_LOCATION: 's3'
  DISCOURSE_S3_BUCKET: 'lc-xyz'
  DISCOURSE_S3_BUCKET_NAME: 'lc-xyz'
  DISCOURSE_S3_BACKUP_BUCKET: 'lc-xyz/backups'
  DISCOURSE_S3_UPLOAD_BUCKET: 'lc-xyz/uploads'
  DISCOURSE_S3_CDN_URL: 'https://lc-rbx.s3.amazonaws.com'

When I include the s3 cdn url the site breaks because all of the links to the assets are on S3 like https://lc-xyz.s3.amazonaws.com/assets/plugin-third-party-01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b.br.js.

Because if you define s3 cdn url Discourse looks for the assets on the s3_cdn_url.

I did a rebuild, but the assets are still missing. I can do a

rake s3:upload_assets

Is there an after_bundling_assets stanza that I could add that to? (I know about after_db_migrate and after_bundle_exec, but don’t know if those would work.)

Do assets get produced some other time? (It would seem like when themes get modified assets would change.)

If I also had a “push CDN like normal”, would that keep this from happening?

Is there some best practice that I’m missing?

2

You can use after_assets_precompile as the hook for that rake task.

3

Aha! Thanks very much for that. Where does the list of those things live?

I’m still confused why this is necessary, especially since you seemed confused before. (Or maybe I should be happy that I can push all this stuff to S3, now that I know how to do it–And I think that it was you who said that one could use CloudFlare’s free CDN to front an AWS bucket.)

4

I’m trying this again. I’ve got uploads going to s3. That works.

I configured cloudflare to CDN the site. If I enter https://lc-XXX.literatehosting.com/uploads in the s3_cdn_url site setting, new uploads go to the S3 bucket and the image is linked to the CDN url and it works.

BUT if I try to set the s3_cdn_url with an ENV in app.yml (or edit config/discourse.conf by hand and sv restart unicorn, all of the assets are loaded from S3 (where they aren’t). That might be OK, but if I try to rake s3:upload_assets, it complains that S3 isn’t configured.

5

I happened across this while figuring out some clashing settings names.

I think (and hope, because it means I understand this and didn’t write absolute nonsense in the thread linked above) that if you remove these two lines:

DISCOURSE_S3_BUCKET: 'lc-xyz'
DISCOURSE_S3_BUCKET_NAME: 'lc-xyz'

Then you can set DISCOURSE_S3_CDN_URL, and it will only be used for uploads, not assets.

6

I think that you do understand, and, thanks to you, I’m at least closer to understanding myself! Thanks very much!

7

Running into this and still very confused after reading this thread…

I’m trying to server s3 uploads (not compiled assets) from CDN (Cloudfront).

If I configure “s3 cdn url” via the settings screen, it works as expected (well…except for System upload not using s3 cdn url)

However, if I configure via DISCOURSE_S3_CDN_URL and rebuild, the frontend is broken because its trying to load compiled assets from my s3 cdn url.

Seems DISCOURSE_S3_CDN_URL / s3_cdn_url should only affect uploads, and DISCOURSE_CDN_URL should only affect assets.

8

That is my experience too. I ended up making a plugin to set the S3_CDN_URL.

9

@pfaffman seems like this should be filed as a bug then, yes?

10

Perhaps. It’s not a feature likely to be used by normal self hosters,so it’s going to be a low priority. Also, I think that there is soon to be a change in how global settings and shadowed by global work, so it’ll likely get worked out then.

11

Sembra lo stesso problema qui, mi hanno respinto.
@pfaffman ho bisogno del tuo aiuto.
Usi CloudFront per ottenere l’URL CDN del dominio personalizzato o ti limiti a usare il prefisso pubblico degli oggetti del bucket nell’URL CDN?

12

Ho appena riscontrato questo problema. @pfaffman, sei riuscito a risolverlo?

Le upload funzionano impostando manualmente s3_cdn_url nelle impostazioni del sito, ma idealmente avrei bisogno di poter impostare la variabile ENV globalmente durante il deployment. Quando lo faccio, incontro lo stesso problema: Discourse cerca gli asset in s3_cdn_url, il che mi sembra un bug in discourse/lib/content_security_policy/default.rb at main · discourse/discourse · GitHub

13

Penso di aver appena impostato il valore nel database.

Il mio parere è che la soluzione consista nel trovare il task Rake che spingerà gli asset su S3.

14

Mi chiedevo se qualcuno del @team potesse confermare se questo sia un comportamento previsto, ovvero che non si possa utilizzare un CDN S3 (impostato globalmente) per i caricamenti senza che Discourse cerchi anche lì le risorse. Sembra un comportamento inaspettato, a meno che non abbia frainteso qualcosa.

15

Sì, questo è il comportamento previsto.

Configurare un CDN S3 lo utilizzerà per tutto ciò che sarebbe un file statico, siano essi upload o asset JS.

16

Ecco alcune informazioni a riguardo, ho dovuto occuparmene lo scorso mese.

L’ho risolto impostando entrambe le variabili (DISCOURSE_S3_CDN_URL e DISCOURSE_CDN_URL) e creando due distribuzioni CloudFront: una per gli upload con origine il bucket S3 e una per le risorse con origine il server.

Ecco il codice che utilizziamo per questo:

Ecco il nostro file app.yml (che abbiamo rinominato web.yml), sostituiamo le variabili al momento della build infra/modules/services/discourse/web.yml at master · debtcollective/infra · GitHub

17

Straordinario. Grazie mille. Non sembra esserci molta documentazione a riguardo e le impostazioni sembrano suggerire che si possa utilizzare S3 solo per i caricamenti, il che penso sia ciò che sta ingannando tutti noi.

18

È nella mia lista scrivere una guida su questo. Spero di farlo entro la fine della settimana.

19

Sì, una volta capito che servono due distribuzioni CloudFront, ha più senso. Inoltre, ricorda di caricare le risorse su S3 dopo ogni ricostruzione. C’è un problema con gli aggiornamenti da Docker Manager: devi eseguire manualmente bundle exec rake s3:upload_assets all’interno del contenitore Docker. Se invece fai una ricostruzione, dovrebbe funzionare se aggiungi queste righe al tuo app.yml

hooks:
  after_assets_precompile:
    - exec:
        cd: $home
        cmd:
          - 'bundle exec rake s3:upload_assets'
20

Quel task sta segnalando che S3 non è configurato. Ho individuato il problema in questo metodo di global_setting.rb:

def self.use_s3?
    (@use_s3 ||=
      begin
        s3_bucket &&
        s3_region && (
          s3_use_iam_profile || (s3_access_key_id && s3_secret_access_key)
        ) ? :true : :false
      end) == :true
  end

Dove ci si aspetta che GlobalSetting.s3_bucket sia definito? A quanto pare dobbiamo impostare le variabili d’ambiente DISCOURSE_S3_UPLOAD_BUCKET e DISCOURSE_S3_BUCKET. Qual è la differenza tra le due?