Redisへの接続エラー

Discourse を起動しようとすると、ブラウザからアクセスできません。ログファイルに原因となる可能性がある情報が見つかりました。どなたか問題の原因をご存知でしょうか?

# /var/discourse/standalone/log/rails/unicorn.stderr.log
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 2 Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) subscribe failed, reconnecting in 1 second. Call stack ["/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:362:in `rescue in establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:343:in `establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:106:in `block in connect'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:306:in `with_reconnect'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:105:in `connect'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:381:in `ensure_connected'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:231:in `block in process'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:319:in `logging'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:230:in `process'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:139:in `block in call_loop'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:292:in `with_socket_timeout'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/client.rb:138:in `call_loop'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/subscribe.rb:43:in `subscription'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis/subscribe.rb:12:in `subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis.rb:3467:in `_subscription'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis.rb:2300:in `block in subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis.rb:52:in `block in synchronize'", "/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis.rb:52:in `synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.3/lib/redis.rb:2299:in `subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-2.2.3/lib/message_bus/backends/redis.rb:287:in `global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-2.2.3/lib/message_bus.rb:721:in `global_subscribe_thread'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-2.2.3/lib/message_bus.rb:669:in `block in new_subscriber_thread'"]
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
# /shared/standalone/log/var-log/redis/current
40:M 11 Dec 2019 17:46:24.868 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
40:M 11 Dec 2019 17:46:24.868 # Server initialized
40:M 11 Dec 2019 17:46:24.868 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
40:M 11 Dec 2019 17:46:24.868 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
40:M 11 Dec 2019 17:46:24.885 * DB loaded from disk: 0.017 seconds
40:M 11 Dec 2019 17:46:24.891 * Ready to accept connections

これらの非常に長いログ行がここで読みづらい場合は、hasteb.in にもアップロードしています: https://hasteb.in/cipemeki.rb

ここで何か奇妙なことが起きています。Docker のアップグレード、サーバーの再起動、Discourse の再構築をお勧めします。

Docker を apt upgrade docker で正常にアップグレードしました(最新バージョンではありませんでした)。reboot でサーバーを再起動し、./launcher rebuild app で再構築を行いました(再構築中にいくつかの Redis 警告が出ましたが、var-log/redis/current ファイル内のものと同様です)。しかし、まだサイトにアクセスできません。Apache2 をプロキシとして使用してアクセスしようとしましたが(ProxyPass)、503 エラーが表示されます。また、IP アドレスと HTTP ポート(HTTPS は無効化済み)でアクセスしようとしましたが、ブラウザから「ERR_CONNECTION_REFUSED」エラーが表示されるだけです。

今回は unicorn.stderr.log に以下のみが出力されています:

I, [2019-12-12T11:58:37.378657 #70]  INFO -- : Refreshing Gem list
I, [2019-12-12T11:58:56.751681 #70]  INFO -- : listening on addr=127.0.0.1:3000 fd=9
I, [2019-12-12T11:59:04.682642 #70]  INFO -- : master process ready
I, [2019-12-12T11:59:23.890076 #172]  INFO -- : worker=0 ready
I, [2019-12-12T11:59:24.852000 #179]  INFO -- : worker=1 ready

また、redis の current ファイルには以前と同じ警告が引き続き出力されています。

これに対する解決策を誰も持っていませんか?まだこれを修正する方法が見つかりません。

明らかなものではありません。/var/discourse/shared/standalone/logs/rails/production.log はどうでしょうか(入力できる場合)?

shared 内に logs フォルダは存在しません。

logs ではなく log です。あなたを助けるためにも、私たちを助けてください…

production.log ファイルは空です。

まだこの問題を解決する方法が見つかりません。Discourse を使っていくつかのフォーラム管理者とも話しましたが、この問題に直面した人はいないようです。

I am also having this issue.
In: unicorn.stdout.log:

Starting up 1 supervised sidekiqs
Loading Sidekiq in process id 4160
2020-01-25T19:29:16.281Z pid=4160 tid=orb7qpo9c ERROR: Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
2020-01-25T19:29:16.288Z pid=4160 tid=orb7qpoj0 ERROR: Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
2020-01-25T19:29:16.287Z pid=4160 tid=orb7rr8k4 ERROR: Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
2020-01-25T19:29:16.283Z pid=4160 tid=orb7qpop4 ERROR: Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

This is brand new Discourse install, standalone container with unix socket connection to outer nginx. I’m trying to embed Discourse in Ghost site for comments, the errors are correlated with displaying a post with a Discourse embed (and Discourse tries to queue a job to parse the referring page?) I had a similar config working for a year but that one blew up trying to upgrade, so I’m reconfiguring from (faulty) memory. Plenty of room for user error here …

Unicorn logs show redis connect error
production.log.txt (155.8 KB) unicorn.stderr.log.txt (14.5 KB)

redis itself seems happy (from standalone/log/var-log/redis/current) current.txt (37.0 KB)
and seems to be handling some connections on the expected port :6379. And some of the connections are from unicorn-worker, FWIW.

root@server3:/var/discourse/shared/standalone/log/var-log/redis# nsenter -t $(docker inspect -f '{{.State.Pid}}' app) -n netstat -ap
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      2250/redis-server * 
tcp        0      0 localhost.localdom:3000 0.0.0.0:*               LISTEN      2280/unicorn master 
tcp        0      0 0.0.0.0:postgresql      0.0.0.0:*               LISTEN      2253/postmaster     
tcp        0      0 localhost.localdo:45404 localhost.localdom:6379 ESTABLISHED 3004/unicorn worker 
tcp        0      0 localhost.localdo:45448 localhost.localdom:6379 ESTABLISHED 2981/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45408 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45346 localhost.localdom:6379 ESTABLISHED 2280/unicorn master 
tcp        0      0 localhost.localdo:45456 localhost.localdom:6379 ESTABLISHED 3013/unicorn worker 
tcp        0      0 localhost.localdom:3000 localhost.localdo:49628 TIME_WAIT   -                   
tcp        0      0 localhost.localdo:45426 localhost.localdom:6379 ESTABLISHED 3022/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45346 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45336 localhost.localdom:6379 ESTABLISHED 2280/unicorn master 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45392 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45458 localhost.localdom:6379 ESTABLISHED 3022/unicorn worker 
tcp        0      0 localhost.localdo:45452 localhost.localdom:6379 ESTABLISHED 3004/unicorn worker 
tcp        0      0 localhost.localdo:45380 localhost.localdom:6379 ESTABLISHED 2981/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45410 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45390 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdo:45374 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdo:45420 localhost.localdom:6379 ESTABLISHED 3013/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45400 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45390 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45332 localhost.localdom:6379 ESTABLISHED 2280/unicorn master 
tcp        0      0 localhost.localdo:45392 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdom:3000 localhost.localdo:49626 TIME_WAIT   -                   
tcp        0      0 localhost.localdom:6379 localhost.localdo:45386 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45428 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45414 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45368 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45332 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45376 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdo:45396 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdom:6379 localhost.localdo:45418 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45448 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45388 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45374 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45408 localhost.localdom:6379 ESTABLISHED 3004/unicorn worker 
tcp        0      0 localhost.localdo:45386 localhost.localdom:6379 ESTABLISHED 2981/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45398 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45398 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdom:6379 localhost.localdo:45430 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45394 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45458 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45456 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45420 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45396 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45394 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdo:45368 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdom:6379 localhost.localdo:45426 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45376 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45428 localhost.localdom:6379 ESTABLISHED 3022/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45404 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45400 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdo:45388 localhost.localdom:6379 ESTABLISHED 2970/sidekiq 6.0.4  
tcp        0      0 localhost.localdo:45414 localhost.localdom:6379 ESTABLISHED 3013/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45336 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45418 localhost.localdom:6379 ESTABLISHED 3013/unicorn worker 
tcp        0      0 localhost.localdo:45384 localhost.localdom:6379 ESTABLISHED 2981/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45384 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdo:45430 localhost.localdom:6379 ESTABLISHED 3022/unicorn worker 
tcp        0      0 localhost.localdo:45410 localhost.localdom:6379 ESTABLISHED 3004/unicorn worker 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45380 ESTABLISHED 2250/redis-server * 
tcp        0      0 localhost.localdom:6379 localhost.localdo:45452 ESTABLISHED 2250/redis-server * 
tcp6       0      0 [::]:6379               [::]:*                  LISTEN      2250/redis-server * 
tcp6       0      0 [::]:postgresql         [::]:*                  LISTEN      2253/postmaster     
udp        0      0 localhost.localdo:57631 localhost.localdo:57631 ESTABLISHED 2253/postmaster     
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name     Path
unix  2      [ ACC ]     STREAM     LISTENING     10681809 2253/postmaster      /var/run/postgresql/.s.PGSQL.5432
unix  3      [ ]         DGRAM                    10681045 2247/rsyslogd        /dev/log
unix  2      [ ACC ]     STREAM     LISTENING     10682768 2257/nginx: master   /shared/nginx.http.sock
unix  3      [ ]         STREAM     CONNECTED     10682773 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682778 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682772 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682776 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682770 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682774 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682775 2257/nginx: master   
unix  2      [ ]         DGRAM                    10681046 2247/rsyslogd        
unix  3      [ ]         STREAM     CONNECTED     10682777 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682779 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10682771 2257/nginx: master   
unix  3      [ ]         STREAM     CONNECTED     10766487 2970/sidekiq 6.0.4   
unix  3      [ ]         STREAM     CONNECTED     10683830 2280/unicorn master  
unix  3      [ ]         STREAM     CONNECTED     10764869 27949/postgres: 10/  /var/run/postgresql/.s.PGSQL.5432
unix  3      [ ]         STREAM     CONNECTED     10684631 2750/postgres: 10/m  /var/run/postgresql/.s.PGSQL.5432
root@server3:/var/discourse/shared/standalone/log/var-log/redis# 

私も同じ問題に直面しています。
ログファイルで localhost:6379 に対して EADDRNOTAVAIL エラーが複数回発生しています。必要であればログを貼ります。

その接続問題は断続的なものですか、それともコンテナの起動時とユニコーンの後に Redis が起動する際のみ発生するものですか?

エラーはログファイル全体に散らばっていますが、再ビルド時には1回だけ表示されます。

私も同じ問題に直面しています!! :sob:

こんにちは、皆さん。

SSH 経由で更新を行いました。
/var/discourse に移動し、./launcher rebuild app を実行しました。

すると、ページが表示されなくなりました。
どうやら Redis に接続できていないようです。

何か心当たりはありますか?

ハンブルグより、
ルネ

ディスク容量を確認しましたか?Redisに接続していないという根拠は何ですか?

Discourse の新しい Docker インストールでも同じエラーが発生しています。最初は正常に動作していましたが、Postgres 12 の設定と静的なメンテナンスエラーページの有効化のために一度停止させました。その後、Discourse を再起動することができなくなりました。

どうやら Rails が Redis に接続できない問題のようです。production.log には以下のようなエラーが記録されています:

Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) subscribe failed, reconnecting in 1 second. Call stack ["/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:363:in `rescue in establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:344:in `establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:106:in `block in connect'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:307:in `with_reconnect'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:105:in `connect'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:382:in `ensure_connected'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:231:in `block in process'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:320:in `logging'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:230:in `process'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis/client.rb:125:in `call'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis.rb:915:in `block in get'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis.rb:51:in `block in synchronize'", "/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis.rb:51:in `synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/redis-4.1.4/lib/redis.rb:914:in `get'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-3.2.0/lib/message_bus/backends/redis.rb:360:in `process_global_backlog'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-3.2.0/lib/message_bus/backends/redis.rb:271:in `block in global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-3.2.0/lib/message_bus/backends/redis.rb:284:in `global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-3.2.0/lib/message_bus.rb:741:in `global_subscribe_thread'", "/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-3.2.0/lib/message_bus.rb:689:in `block in new_subscriber_thread'"]
Creating scope :open. Overwriting existing method Poll.open.
Creating scope :open. Overwriting existing method Poll.open.
Creating scope :open. Overwriting existing method Poll.open.

production_errors.log は空です。Redis は通常通り起動しているようです。ただし、syslog に以下の行が含まれている点に注意が必要です:

May 17 13:35:30 wobbly-wiki-forum-app rsyslogd: imklog: cannot open kernel log (/proc/kmsg): Operation not permitted.
May 17 13:35:30 wobbly-wiki-forum-app rsyslogd: activation of module imklog failed [v8.1901.0 try https://www.rsyslog.com/e/2145 ]
May 17 13:35:30 wobbly-wiki-forum-app rsyslogd:  [origin software="rsyslogd" swVersion="8.1901.0" x-pid="51" x-info="https://www.rsyslog.com"] start

これらが何かの問題を示しているのでしょうか?参考までに、サーバーには十分なメモリとディスク容量があります。

編集:解決しました!nginx リバースプロキシ経由で実行していましたが、手順 を完全に守っていなかったことが原因でした。テンプレート "templates/web.socketed.template.yml" を追加することを忘れており、また app.ymlexpose: セクション全体をコメントアウトしていました。これで正常に動作するようになりました。

他に解決策を見つけた方はいらっしゃいますか? Ubuntu Server 20.04 のバニラ Docker ベースのインストールでこのエラーが発生しています。Nginx リバースプロキシ経由で何も実行していません。デフォルトの Docker インストールスクリプトを使用しただけです。