Container Discourse com UnixSocket para Redis?

[ryanerwin]

Curious if anyone is using unixsocket /var/discourse/shared/... for Redis in their Standalone container? Seems redis-rb also supports domain sockets with:

redis = Redis.new(path: "/var/discourse/shared/standalone/redis.sock")

Apparently connecting via the Unix Domain Socket is about 20% faster than using TCP sockets, according to this article on medium…

Also, using Unix Domain Sockets would make it easier to run multiple standalone containers without separating your web and data containers… Otherwise I think you’ll end up with conflicts with Redis listening on 127.0.0.1…

I’m trying to setup two completely standalone containers on one host right now, and since they’re both for very small sites, I would prefer the flexibility of keeping them in standalone containers… Unfortunately Redis listening on 127.0.0.1 (and presumably Postgres too) will conflict…

[neounix (Dark Matter)]

@ryanerwin

Also, using Unix Domain Sockets would make it easier to run multiple standalone containers without separating your web and data containers… Otherwise I think you’ll end up with conflicts with Redis listening on 127.0.0.1…

Hi @ryanerwin

Seems you are not fully understanding containers and docker a bit; so let me help you out.

Redis runs by default in the standalone container on port 6379:

cd /var/discourse
./launcher enter app
apt install net-tools
netstat -an | grep :6379 |wc -l 

74

Now lets exit the container and check netstat for Redis:

exit
root@localhost:/var/discourse# netstat -an | grep :6379 |wc -l 
0

So, you can see that Redis is listening on localhost inside the container; and localhost inside the container was (is) not exposed outside the container.

Hence, if you have many standalone discourse containers running, there would not be any Redis conflicts between containers because Redis has not been exposed outside each container.

That’s why it’s called a “container”…

Every socket inside the container must be explicitly exposed to be available outside the container.

Hope this helps in some small way.

---

Note that unix domain sockets are very cool… I only addressed your comment about perceived Redis conflicts between standalone containers and did not address the unix domain socket topic.

[ryanerwin]

@neounix

Hence, if you have many standalone discourse containers running, there would not be any Redis conflicts between containers because Redis has not been exposed outside each container.

That’s how I thought it would work with discourse running inside of Docker too, however when I actually ran it, during the bootstrapping process I saw:

INFO -- : > cd /var/www/discourse && git reset --hard
# oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
# Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=195, just started
# Configuration loaded
# Could not create server TCP listening socket *:6379: bind: Address already in use
Checking out files: 100% (27893/27893), done.

And I had found this thread about “install fails because of other redis container”, however the problem actually was low disk space… Reorganized some things and it does work just fine with multiple standalone containers.

[neounix (Dark Matter)]

@ryanerwin

Reorganized some things and it does work just fine with multiple standalone containers.

Dear @ryanerwin,

It’s great to read that you have found the underlying issue and now understand that Redis runs “in the container” and is not exposed (from a socket I/O perspective) outside the container (as set up OOTB).

Regarding using unix domain sockets and Redis, I think this is a great idea and have not set this up (yet), and nor have I read any write-up on how to set this up for Discourse, and so I would encourage you to continue to explore this option.

If I have time, I will research this further and try to set up Redis to use a unix domain docket in Discourse on a staging server. In the meantime, if you can work this out and post your results, that would be much appreciated. I’m sure many others are also interested in this interesting Redis topic.

Thanks.

[neounix (Dark Matter)]

@ryanerwin

Curious if anyone is using unixsocket /var/discourse/shared/... for Redis in their Standalone container? Seems redis-rb also supports domain sockets with:

Hi @ryanerwin,

I hope you will be happy to hear that I have Discourse running with Redis using a unix socket in a standalone container, just like you asked about.

Check out the bottom of this screen shot from sidekiq:

In addition, here are some more screenshots from building the app:

My next steps are to:

1. Move the unix socket to the shared volume so it can be accessed outside the container.
2. Retesting with minimal ENV vars to get the “bare bones” changes to get this up and running.

Basically, I have created this new template:

-rw-r–r-- 1 root root 2028 Jun 6 08:13 redis.socketed.template.yml

and made minor changes to old friend app.yml

Since I only started on this today, I plan to test a bit more before publishing the details.

Hope this will be helpful.

---

Update

---

Works as expected outside the container when the unix socket is on a shared volume:

[neounix (Dark Matter)]

PR for this:

https://github.com/discourse/discourse_docker/pull/469

Note:

To implement:

• Change the redis template in the container yml file
• Add one additional line to the same container yml file

 ## Set the REDIS_URL and use the redis.socketed.template.yml to use 
 ## a unix domain socket for Redis
 REDIS_URL: unix:///shared/tmp/redis.sock

Implementation Notes:

1. If concerned about security of the Redis DB on the host, no need to expose this unix socket in the shared volume.

2. If you wish to set the permissions of the unix socket to 770 (instead of 777) , change the group of the unix socket to www-data.

[MichaIng]

Alguém mais percebeu que REDIS_URL não tem mais efeito? Durante a (re)construção e também na inicialização do contêiner, apesar de REDIS_URL estar configurado para conectar via socket UNIX, ele tenta conectar via redis://localhost:6379.

Por um lado, faz sentido, já que o Discourse tem configurações para e aplica apenas host e porta: discourse/app/models/global_setting.rb at main · discourse/discourse · GitHub
Haveria um parâmetro path para definir um caminho de socket UNIX, que substituiria host e porta. E haveria um parâmetro url, para definir uma URL completa como unix:///shared/redis_data/redis.sock.

O gem Redis documenta que a variável de ambiente REDIS_URL substituiria quaisquer outras configurações, e até uma certa versão do Discourse/Redis gem isso funcionou: redis-rb/lib/redis.rb at master · redis/redis-rb · GitHub

O uso do socket UNIX quebrou quando o Discourse mudou para a versão 5 do gem Redis: DEV: Upgrade the Redis gem to v5.4 · discourse/discourse@2ed31fe · GitHub
Então, imagino que REDIS_URL quebrou no gem Redis (não substituindo mais outras opções), e não pelo Discourse (onde o código relacionado não mudou)?

Provavelmente este commit quebrou: Use redis-client as transport · redis/redis-rb@08a2100 · GitHub
Eu reportaria no repositório do gem Redis, ou alguém sabe se é um problema com a forma como o Discourse o implementa?

A variável de ambiente, aliás, é passada corretamente. Eu a deixei no lugar e apenas configurei o Redis para não escutar no socket UNIX, e enquanto o unicorn inicia bem, o sidekiq falha, sentindo falta do socket UNIX em vez disso :

Error in demon processes heartbeat check: No such file or directory - connect(2) for /shared/redis_data/redis.sock
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:116:in `initialize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:116:in `new'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:116:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/ruby_connection.rb:51:in `initialize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:759:in `new'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:759:in `block in connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client/middlewares.rb:12:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:758:in `connect'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:745:in `raw_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:705:in `ensure_connected'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/redis-client-0.24.0/lib/redis_client.rb:285:in `call'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/redis_client_adapter.rb:36:in `block (2 levels) in <module:CompatMethods>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/api.rb:912:in `block in cleanup'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/config.rb:175:in `block in redis'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:110:in `block (2 levels) in with'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:109:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:109:in `block in with'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:106:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/connection_pool-2.5.3/lib/connection_pool.rb:106:in `with'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/config.rb:172:in `redis'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq.rb:74:in `redis'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/api.rb:912:in `cleanup'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/sidekiq-7.3.9/lib/sidekiq/api.rb:903:in `initialize'
/var/www/discourse/lib/demon/sidekiq.rb:25:in `new'
/var/www/discourse/lib/demon/sidekiq.rb:25:in `heartbeat_check'
config/unicorn.conf.rb:131:in `block (2 levels) in reload'

Isso se encaixa no quadro de que REDIS_URL ainda funciona, mas apenas se as configurações de conexão não forem definidas de outra forma. Olhando o rastreamento de erro, global_setting.rb não aplica host e porta para o sidekiq, como faz para o unicorn.