Decoupled Discourse Application - Managed Redis, Managed Postgres, and DIgital Ocean Volume with Discourse

hosting
1

I’m trying to connect Discourse to Redis on Digital Ocean Managed Redis Database. I was able to get the postgres working fine with Digital Ocean Managed Postgres Database, but Redis give me an error. What can you make of this?

    templates:
      - templates/web.template.yml
      - templates/web.ssl.template.yml
    # Use 'links' key to link containers together, aka use Docker --link flag.
    expose:
      - '80:80'
      - '443:443'
    params:
      db_default_text_search_config: pg_catalog.english
      db_shared_buffers: 512MB
    env:
      LANG: en_US.UTF-8
      UNICORN_WORKERS: 8
    
      # https://github.com/discourse/discourse/blob/master/config/discourse_defaults.conf
      DISCOURSE_HOSTNAME: blah.example.com
      DISCOURSE_DEVELOPER_EMAILS: email@example.com
      DISCOURSE_SMTP_ADDRESS: smtp.mailgun.org
      DISCOURSE_SMTP_PORT: 587
      DISCOURSE_SMTP_USER_NAME: blah@mail.com
      DISCOURSE_SMTP_PASSWORD: 9cd16e-aff2d1b9-36c86fff
      DISCOURSE_DB_NAME: defaultdb
      DISCOURSE_DB_USERNAME: doadmin
      DISCOURSE_DB_PASSWORD: gp5m224
      DISCOURSE_DB_HOST: private-digitalocean-stage-discuss-postgres-do-user-.ondigitalocean.com
      DISCOURSE_DB_PORT: 25060
      DISCOURSE_REDIS_USERNAME: default
      DISCOURSE_REDIS_PASSWORD: dex3mf
      DISCOURSE_REDIS_HOST: private-digitalocean-stage-discuss-redis-do-user-966537-0.b.db.ondigitalocean.com
      DISCOURSE_REDIS_PORT: 25061
      DISCOURSE_REDIS_CLIENT_ID: ~
    volumes:
      - volume:
          host: /var/discourse/shared/standalone
          guest: /shared
      - volume:
          host: /var/discourse/shared/standalone/log/var-log
          guest: /var/log
    hooks:
      after_code:
        - exec:
            cd: $home/plugins
            cmd:
              - 'git clone https://github.com/discourse/docker_manager.git'
    run:
      - exec: echo "Beginning of custom commands"
      - exec: echo "End of custom commands"

See here, I added:

  DISCOURSE_REDIS_USERNAME: default
  DISCOURSE_REDIS_PASSWORD: dex3mf
  DISCOURSE_REDIS_HOST: private-digitalocean-stage-discuss-redis-do-user-.db.ondigitalocean.com
  DISCOURSE_REDIS_PORT: 25061
  DISCOURSE_REDIS_CLIENT_ID: ~

But I get this error:

  I, [2021-02-13T04:05:48.508236 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
  Failed to report error: Connection lost (ECONNRESET) 2 Connection lost (ECONNRESET) subscribe failed, reconnecting in 1 second. Call stack /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:275:in `rescue in io'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:267:in `io'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:279:in `read'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:131:in `block in call'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:248:in `block (2 levels) in process'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:389:in `ensure_connected'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:238:in `block in process'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:325:in `logging'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:237:in `process'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:131:in `call'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:113:in `block in connect'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:313:in `with_reconnect'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:111:in `connect'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:294:in `with_socket_timeout'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:144:in `call_loop'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/subscribe.rb:44:in `subscription'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/subscribe.rb:14:in `subscribe'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:3507:in `_subscription'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:2326:in `block in subscribe'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `block in synchronize'
    /usr/local/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
    /usr/local/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `synchronize'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:2325:in `subscribe'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus/backends/redis.rb:287:in `global_subscribe'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus.rb:766:in `global_subscribe_thread'
    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus.rb:714:in `block in new_subscriber_thread'
    rake aborted!
2

I worked to ensure we are compatible with all managed services from Digital Ocean a while back. Their Redis has mandatory SSL which we did not support at the time.

To get their Redis to work use

  DISCOURSE_REDIS_HOST: falcoland-redis-do-user-435229-0.a.db.ondigitalocean.com
  DISCOURSE_REDIS_PASSWORD: vp39d0dpy8dxn68n
  DISCOURSE_REDIS_PORT: 25061
  DISCOURSE_REDIS_USE_SSL: true
4 Likes
3

Thanks @Falco ~~ works!!!

My next step is to move the /var/discourse to a Digital Ocean volume that the Discourse application servers can share. Would you just mount the Volume and symlink the directories?

My target architecture is Load Balancer → multiple discourse servers with DO Redis and PostGres, and one shared DO volume that the multiple discourse servers share rather than having their own volumes.

What you think?

4

Why?

Put all static assets and user uploads on Digital Ocean Spaces following Using Object Storage for Uploads (S3 & Clones) and your app servers won’t need to share anything.

One thing to keep in mind is that with multiple app droplets, managed Redis, PostgreSQL and Object Storage the price gets pretty close to our managed hosting :wink:

4 Likes
5

thanks, my forum does not use the frontend templates. I have an api built in NodeJS and the frontend is VueJS. So I’m only using discourse to share and store data. I don’t allow image uploads in my forum.

So this is my use case, all discourse application servers can share one volume of files behind the load balancer. Each piece is now decoupled except the /var/discourse volume. Make sense?

6

Hey @Falco , I figured it out pretty easy:

First, make the volume in DO and ensure you link your droplet to it in the DO steps, then
ssh into your discourse machine

mv /var/discourse /var/discourse.bak
mkdir /home/discourse
mount /dev/sda /home/discourse
mv /var/discourse.bak/* /home/discourse
nano /home/discourse/containers/app.yml

Update the volumes section:

volumes:
  - volume:
      host: /home/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /home/discourse/shared/standalone/log/var-log
      guest: /var/log

then,

cd /home/discourse
sudo ./launcher rebuild app

Now your discourse is served from volume, it can take snapshot and deployed behind load balancer since all parts are now decoupled.

cheers!

2 Likes