Discourse Container with UnixSocket for Redis?

[ryanerwin]

Curious if anyone is using unixsocket /var/discourse/shared/... for Redis in their Standalone container? Seems redis-rb also supports domain sockets with:

redis = Redis.new(path: "/var/discourse/shared/standalone/redis.sock")

Apparently connecting via the Unix Domain Socket is about 20% faster than using TCP sockets, according to this article on medium…

Also, using Unix Domain Sockets would make it easier to run multiple standalone containers without separating your web and data containers… Otherwise I think you’ll end up with conflicts with Redis listening on 127.0.0.1…

I’m trying to setup two completely standalone containers on one host right now, and since they’re both for very small sites, I would prefer the flexibility of keeping them in standalone containers… Unfortunately Redis listening on 127.0.0.1 (and presumably Postgres too) will conflict…

[neounix (Dark Matter)]

@ryanerwin

Also, using Unix Domain Sockets would make it easier to run multiple standalone containers without separating your web and data containers… Otherwise I think you’ll end up with conflicts with Redis listening on 127.0.0.1…

Hi @ryanerwin

Seems you are not fully understanding containers and docker a bit; so let me help you out.

Redis runs by default in the standalone container on port 6379:

cd /var/discourse
./launcher enter app
apt install net-tools
netstat -an | grep :6379 |wc -l 

74

Now lets exit the container and check netstat for Redis:

exit
root@localhost:/var/discourse# netstat -an | grep :6379 |wc -l 
0

So, you can see that Redis is listening on localhost inside the container; and localhost inside the container was (is) not exposed outside the container.

Hence, if you have many standalone discourse containers running, there would not be any Redis conflicts between containers because Redis has not been exposed outside each container.

That’s why it’s called a “container”…

Every socket inside the container must be explicitly exposed to be available outside the container.

Hope this helps in some small way.

---

Note that unix domain sockets are very cool… I only addressed your comment about perceived Redis conflicts between standalone containers and did not address the unix domain socket topic.

[ryanerwin]

@neounix

Hence, if you have many standalone discourse containers running, there would not be any Redis conflicts between containers because Redis has not been exposed outside each container.

That’s how I thought it would work with discourse running inside of Docker too, however when I actually ran it, during the bootstrapping process I saw:

INFO -- : > cd /var/www/discourse && git reset --hard
# oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
# Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=195, just started
# Configuration loaded
# Could not create server TCP listening socket *:6379: bind: Address already in use
Checking out files: 100% (27893/27893), done.

And I had found this thread about “install fails because of other redis container”, however the problem actually was low disk space… Reorganized some things and it does work just fine with multiple standalone containers.

[neounix (Dark Matter)]

@ryanerwin

Reorganized some things and it does work just fine with multiple standalone containers.

Dear @ryanerwin,

It’s great to read that you have found the underlying issue and now understand that Redis runs “in the container” and is not exposed (from a socket I/O perspective) outside the container (as set up OOTB).

Regarding using unix domain sockets and Redis, I think this is a great idea and have not set this up (yet), and nor have I read any write-up on how to set this up for Discourse, and so I would encourage you to continue to explore this option.

If I have time, I will research this further and try to set up Redis to use a unix domain docket in Discourse on a staging server. In the meantime, if you can work this out and post your results, that would be much appreciated. I’m sure many others are also interested in this interesting Redis topic.

Thanks.

[neounix (Dark Matter)]

@ryanerwin

Curious if anyone is using unixsocket /var/discourse/shared/... for Redis in their Standalone container? Seems redis-rb also supports domain sockets with:

Hi @ryanerwin,

I hope you will be happy to hear that I have Discourse running with Redis using a unix socket in a standalone container, just like you asked about.

Check out the bottom of this screen shot from sidekiq:

In addition, here are some more screenshots from building the app:

My next steps are to:

1. Move the unix socket to the shared volume so it can be accessed outside the container.
2. Retesting with minimal ENV vars to get the “bare bones” changes to get this up and running.

Basically, I have created this new template:

-rw-r–r-- 1 root root 2028 Jun 6 08:13 redis.socketed.template.yml

and made minor changes to old friend app.yml

Since I only started on this today, I plan to test a bit more before publishing the details.

Hope this will be helpful.

---

Update

---

Works as expected outside the container when the unix socket is on a shared volume:

[neounix (Dark Matter)]

PR for this:

https://github.com/discourse/discourse_docker/pull/469

Note:

To implement:

• Change the redis template in the container yml file
• Add one additional line to the same container yml file

 ## Set the REDIS_URL and use the redis.socketed.template.yml to use 
 ## a unix domain socket for Redis
 REDIS_URL: unix:///shared/tmp/redis.sock

Implementation Notes:

1. If concerned about security of the Redis DB on the host, no need to expose this unix socket in the shared volume.

2. If you wish to set the permissions of the unix socket to 770 (instead of 777) , change the group of the unix socket to www-data.

[MichaIng]

Did anyone else recognize that REDIS_URL has no effect anymore? During (re)build and as well at container start, despite REDIS_URL set to connect via UNIX socket, it tries to connect to via redis://localhost:6379.

On the one hand, it makes sense, since Discourse has configs for and applies host and port only: discourse/app/models/global_setting.rb at main · discourse/discourse · GitHub
There would be a path parameter to define a UNIX socket path, which would override host and port. And there would be a url parameter, to define a full URL like unix:///shared/redis_data/redis.sock.

The Redis gem documents that the REDIS_URL environment variable would override any other settings, and until a certain Discourse/Redis gem version this worked: redis-rb/lib/redis.rb at master · redis/redis-rb · GitHub

UNIX socket usage broke when Discourse moved to the Redis gems version 5: DEV: Upgrade the Redis gem to v5.4 · discourse/discourse@2ed31fe · GitHub
So I guess REDIS_URL broke in the Redis gem (not overriding other options anymore), and not by Discourse (where the related code did not change)?

Likely this commit broke it: Use redis-client as transport · redis/redis-rb@08a2100 · GitHub
I would report it at the Redis gem repo, or does someone know it is an issue with how Discourse implements it?

The environment variable btw is passed correctly. I left it in place and only configured Redis to not listen on the UNIX socket, and while unicorn starts up nicely, sidekiq fails, missing the UNIX socket instead :

Error in demon processes heartbeat check: No such file or directory - connect(2) for /shared/redis_data/redis.sock
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:116:in `initialize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:116:in `new'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:116:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:51:in `initialize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:759:in `new'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:759:in `block in connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/middlewares.rb:12:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:758:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:745:in `raw_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:705:in `ensure_connected'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:285:in `call'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/redis_client_adapter.rb:36:in `block (2 levels) in <module:CompatMethods>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/api.rb:912:in `block in cleanup'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/config.rb:175:in `block in redis'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:110:in `block (2 levels) in with'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:109:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:109:in `block in with'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:106:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:106:in `with'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/config.rb:172:in `redis'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq.rb:74:in `redis'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/api.rb:912:in `cleanup'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/api.rb:903:in `initialize'
/var/www/discourse/lib/demon/sidekiq.rb:25:in `new'
/var/www/discourse/lib/demon/sidekiq.rb:25:in `heartbeat_check'
config/unicorn.conf.rb:131:in `block (2 levels) in reload'

It fits into the picture that REDIS_URL still works, but only if connection settings are not defined otherwise. Looking at the error trace, global_setting.rb does not apply host and port for sidekiq, like it does for unicorn.