Definir enlaces DISCOURSE_S3_CDN_URL a activos en la URL de CDN de S3

Soporte
1

I had this problem before and decided that I was crazy, confused, or the database on the site was suspect, but this is on a brand new site. Also, I was on Digital Ocean spaces, so I thought it might be a problem somehow.

I’m trying again to figure out how to keep images on AWS S3 like I think the Big Boys do.

Here’s what I have in the env section:

  DISCOURSE_S3_ACCESS_KEY_ID: 'key'
  DISCOURSE_S3_SECRET_ACCESS_KEY: 'lock'
  DISCOURSE_BACKUP_LOCATION: 's3'
  DISCOURSE_S3_BUCKET: 'lc-xyz'
  DISCOURSE_S3_BUCKET_NAME: 'lc-xyz'
  DISCOURSE_S3_BACKUP_BUCKET: 'lc-xyz/backups'
  DISCOURSE_S3_UPLOAD_BUCKET: 'lc-xyz/uploads'
  DISCOURSE_S3_CDN_URL: 'https://lc-rbx.s3.amazonaws.com'

When I include the s3 cdn url the site breaks because all of the links to the assets are on S3 like https://lc-xyz.s3.amazonaws.com/assets/plugin-third-party-01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b.br.js.

Because if you define s3 cdn url Discourse looks for the assets on the s3_cdn_url.

I did a rebuild, but the assets are still missing. I can do a

rake s3:upload_assets

Is there an after_bundling_assets stanza that I could add that to? (I know about after_db_migrate and after_bundle_exec, but don’t know if those would work.)

Do assets get produced some other time? (It would seem like when themes get modified assets would change.)

If I also had a “push CDN like normal”, would that keep this from happening?

Is there some best practice that I’m missing?

2

You can use after_assets_precompile as the hook for that rake task.

3

Aha! Thanks very much for that. Where does the list of those things live?

I’m still confused why this is necessary, especially since you seemed confused before. (Or maybe I should be happy that I can push all this stuff to S3, now that I know how to do it–And I think that it was you who said that one could use CloudFlare’s free CDN to front an AWS bucket.)

4

I’m trying this again. I’ve got uploads going to s3. That works.

I configured cloudflare to CDN the site. If I enter https://lc-XXX.literatehosting.com/uploads in the s3_cdn_url site setting, new uploads go to the S3 bucket and the image is linked to the CDN url and it works.

BUT if I try to set the s3_cdn_url with an ENV in app.yml (or edit config/discourse.conf by hand and sv restart unicorn, all of the assets are loaded from S3 (where they aren’t). That might be OK, but if I try to rake s3:upload_assets, it complains that S3 isn’t configured.

5

I happened across this while figuring out some clashing settings names.

I think (and hope, because it means I understand this and didn’t write absolute nonsense in the thread linked above) that if you remove these two lines:

DISCOURSE_S3_BUCKET: 'lc-xyz'
DISCOURSE_S3_BUCKET_NAME: 'lc-xyz'

Then you can set DISCOURSE_S3_CDN_URL, and it will only be used for uploads, not assets.

6

I think that you do understand, and, thanks to you, I’m at least closer to understanding myself! Thanks very much!

7

Running into this and still very confused after reading this thread…

I’m trying to server s3 uploads (not compiled assets) from CDN (Cloudfront).

If I configure “s3 cdn url” via the settings screen, it works as expected (well…except for System upload not using s3 cdn url)

However, if I configure via DISCOURSE_S3_CDN_URL and rebuild, the frontend is broken because its trying to load compiled assets from my s3 cdn url.

Seems DISCOURSE_S3_CDN_URL / s3_cdn_url should only affect uploads, and DISCOURSE_CDN_URL should only affect assets.

8

That is my experience too. I ended up making a plugin to set the S3_CDN_URL.

9

@pfaffman seems like this should be filed as a bug then, yes?

10

Perhaps. It’s not a feature likely to be used by normal self hosters,so it’s going to be a low priority. Also, I think that there is soon to be a change in how global settings and shadowed by global work, so it’ll likely get worked out then.

11

Parece que el mismo problema me está rechazando aquí
@pfaffman necesito tu ayuda
¿Usas CloudFront para lograr la URL de CDN de dominio personalizado o simplemente usas el prefijo público de los objetos del bucket en la URL de CDN?

12

Acabo de encontrarme con esto. @pfaffman, ¿lograste resolverlo?

He logrado que las subidas funcionen configurando manualmente s3_cdn_url en la configuración del sitio, pero idealmente necesito poder establecer la variable de entorno de forma global al desplegar. Cuando lo hago, obtengo el mismo problema: Discourse intenta buscar los recursos en s3_cdn_url, lo cual me parece un error en discourse/lib/content_security_policy/default.rb at main · discourse/discourse · GitHub

13

Creo que acabo de establecer el valor en la base de datos.

Mi suposición es que la solución consiste en encontrar la tarea de Rake que subirá los activos a S3.

14

Me preguntaba si alguno de los @team podría confirmar si este es el comportamiento esperado, es decir, que no se puede usar un CDN de S3 (configurado globalmente) para las cargas sin que Discourse también busque los recursos allí. Parece un comportamiento inesperado, a menos que esté malinterpretando algo.

15

Sí, ese es el comportamiento.

Configurar un CDN de S3 lo utilizará para todo lo que sea un archivo estático, ya sean cargas o activos de JS.

16

Aquí tienes información al respecto; estuve lidiando con esto el mes pasado.

Lo solucioné configurando ambas variables (DISCOURSE_S3_CDN_URL y DISCOURSE_CDN_URL) y creando dos distribuciones de CloudFront: una para las subidas con el bucket S3 como origen, y otra para los activos con el servidor como origen.

Aquí está el código que utilizamos para esto:

Este es nuestro app.yml (lo llamamos web.yml); reemplazamos las variables en el momento de la compilación infra/modules/services/discourse/web.yml at master · debtcollective/infra · GitHub

17

Increíble. Muchas gracias. No parece haber mucha documentación al respecto en absoluto, y la configuración sugiere que solo puedes usar S3 para cargas, lo cual creo que es lo que nos está confundiendo a todos.

18

Tengo en mi lista escribir un howto sobre esto. Espero hacerlo para el fin de semana.

19

Sí, una vez que entiendes que necesitas dos distribuciones de CloudFront, tiene más sentido. También, recuerda subir los activos a S3 después de cada reconstrucción. Hay un problema con las actualizaciones desde Docker Manager donde necesitas ejecutar bundle exec rake s3:upload_assets manualmente dentro del contenedor Docker. Si haces una reconstrucción en su lugar, debería funcionar si agregas estas líneas a tu app.yml

hooks:
  after_assets_precompile:
    - exec:
        cd: $home
        cmd:
          - 'bundle exec rake s3:upload_assets'
20

Esa tarea está indicando que no tengo S3 configurado. He reducido el problema a este método en global_setting.rb:

def self.use_s3?
    (@use_s3 ||=
      begin
        s3_bucket &&
        s3_region && (
          s3_use_iam_profile || (s3_access_key_id && s3_secret_access_key)
        ) ? :true : :false
      end) == :true
  end

¿Dónde se espera que esté definido GlobalSetting.s3_bucket? Por lo que veo, necesitamos establecer las variables de entorno DISCOURSE_S3_UPLOAD_BUCKET y DISCOURSE_S3_BUCKET. ¿Cuál es la diferencia entre ambas?