502 Bad Gateway apenas para dois Tópicos

niccokunzmann · Outubro 26, 2022, 4:24pm

I have a 502 Bad gateway for a topic (maybe more than one) on the discourse forum I host:

https://ammanvalley.foss.wales/t/wip-arduino-workshop/58

I just document my way here and see if someone can help.

I did

./launcher rebuild app - Discourse stuck on "Currently Upgrading" - #2 by sam
upgrading all and canceling the stuck upgrade
- Stuck on “currently upgrading...”
- "Currently upgrading" even after a rebuild

I wonder what to do. I will search a bit more.

restarting the docker container - "Currently upgrading" even after a rebuild - #3 by codinghorror

HEAD requests

Here is a list of pages:

$ for i in `seq 100`; do ( status="`curl -L --head -o /dev/null -s -w '%{http_code}' https://ammanvalley.foss.wales/t/$i`"; echo "https://ammanvalley.foss.wales/t/$i - $status" ) done
https://ammanvalley.foss.wales/t/1 - 200
https://ammanvalley.foss.wales/t/2 - 404
https://ammanvalley.foss.wales/t/3 - 200
https://ammanvalley.foss.wales/t/4 - 404
https://ammanvalley.foss.wales/t/5 - 404
https://ammanvalley.foss.wales/t/6 - 404
https://ammanvalley.foss.wales/t/7 - 200 -- no 502
https://ammanvalley.foss.wales/t/8 - 404
https://ammanvalley.foss.wales/t/9 - 404
https://ammanvalley.foss.wales/t/10 - 200
https://ammanvalley.foss.wales/t/11 - 200
https://ammanvalley.foss.wales/t/12 - 200
https://ammanvalley.foss.wales/t/13 - 200
https://ammanvalley.foss.wales/t/14 - 200
https://ammanvalley.foss.wales/t/15 - 200
https://ammanvalley.foss.wales/t/16 - 404
https://ammanvalley.foss.wales/t/17 - 404
https://ammanvalley.foss.wales/t/18 - 200
https://ammanvalley.foss.wales/t/19 - 200
https://ammanvalley.foss.wales/t/20 - 404
https://ammanvalley.foss.wales/t/21 - 200
https://ammanvalley.foss.wales/t/22 - 200
https://ammanvalley.foss.wales/t/23 - 200
https://ammanvalley.foss.wales/t/24 - 200
https://ammanvalley.foss.wales/t/25 - 200
https://ammanvalley.foss.wales/t/26 - 404
https://ammanvalley.foss.wales/t/27 - 404
https://ammanvalley.foss.wales/t/28 - 200
https://ammanvalley.foss.wales/t/29 - 404
https://ammanvalley.foss.wales/t/30 - 200
https://ammanvalley.foss.wales/t/31 - 200
https://ammanvalley.foss.wales/t/32 - 200
https://ammanvalley.foss.wales/t/33 - 200
https://ammanvalley.foss.wales/t/34 - 200
https://ammanvalley.foss.wales/t/35 - 200
https://ammanvalley.foss.wales/t/36 - 404
https://ammanvalley.foss.wales/t/37 - 200
https://ammanvalley.foss.wales/t/38 - 200
https://ammanvalley.foss.wales/t/39 - 200
https://ammanvalley.foss.wales/t/40 - 404
https://ammanvalley.foss.wales/t/41 - 200
https://ammanvalley.foss.wales/t/42 - 200
https://ammanvalley.foss.wales/t/43 - 200
https://ammanvalley.foss.wales/t/44 - 200
https://ammanvalley.foss.wales/t/45 - 200
https://ammanvalley.foss.wales/t/46 - 200
https://ammanvalley.foss.wales/t/47 - 404
https://ammanvalley.foss.wales/t/48 - 200
https://ammanvalley.foss.wales/t/49 - 200
https://ammanvalley.foss.wales/t/50 - 200
https://ammanvalley.foss.wales/t/51 - 200
https://ammanvalley.foss.wales/t/52 - 200
https://ammanvalley.foss.wales/t/53 - 200
https://ammanvalley.foss.wales/t/54 - 200
https://ammanvalley.foss.wales/t/55 - 200
https://ammanvalley.foss.wales/t/56 - 200
https://ammanvalley.foss.wales/t/57 - 200
https://ammanvalley.foss.wales/t/58 - 200 -- no 502
https://ammanvalley.foss.wales/t/59 - 200
https://ammanvalley.foss.wales/t/60 - 200
https://ammanvalley.foss.wales/t/61 - 200
https://ammanvalley.foss.wales/t/62 - 404
https://ammanvalley.foss.wales/t/63 - 200
https://ammanvalley.foss.wales/t/64 - 404
https://ammanvalley.foss.wales/t/65 - 404
https://ammanvalley.foss.wales/t/66 - 200
https://ammanvalley.foss.wales/t/67 - 200
https://ammanvalley.foss.wales/t/68 - 404
https://ammanvalley.foss.wales/t/69 - 200
https://ammanvalley.foss.wales/t/70 - 404
https://ammanvalley.foss.wales/t/71 - 404
https://ammanvalley.foss.wales/t/72 - 404
https://ammanvalley.foss.wales/t/73 - 200
https://ammanvalley.foss.wales/t/74 - 404
https://ammanvalley.foss.wales/t/75 - 404
https://ammanvalley.foss.wales/t/76 - 404
https://ammanvalley.foss.wales/t/77 - 404
https://ammanvalley.foss.wales/t/78 - 404
https://ammanvalley.foss.wales/t/79 - 404
https://ammanvalley.foss.wales/t/80 - 404
https://ammanvalley.foss.wales/t/81 - 404
https://ammanvalley.foss.wales/t/82 - 404
https://ammanvalley.foss.wales/t/83 - 404
https://ammanvalley.foss.wales/t/84 - 404
https://ammanvalley.foss.wales/t/85 - 404
https://ammanvalley.foss.wales/t/86 - 404
https://ammanvalley.foss.wales/t/87 - 404
https://ammanvalley.foss.wales/t/88 - 404
https://ammanvalley.foss.wales/t/89 - 404
https://ammanvalley.foss.wales/t/90 - 404
https://ammanvalley.foss.wales/t/91 - 404
https://ammanvalley.foss.wales/t/92 - 404
https://ammanvalley.foss.wales/t/93 - 404
https://ammanvalley.foss.wales/t/94 - 404
https://ammanvalley.foss.wales/t/95 - 404
https://ammanvalley.foss.wales/t/96 - 404
https://ammanvalley.foss.wales/t/97 - 404
https://ammanvalley.foss.wales/t/98 - 404
https://ammanvalley.foss.wales/t/99 - 404
https://ammanvalley.foss.wales/t/100 - 404

GET requests

$ for i in `seq 100`; do ( status="`curl -L -o /dev/null -s -w '%{http_code}' https://ammanvalley.foss.wales/t/$i`"; echo "https://ammanvalley.foss.wales/t/$i - $status" ) done
https://ammanvalley.foss.wales/t/1 - 200
https://ammanvalley.foss.wales/t/2 - 404
https://ammanvalley.foss.wales/t/3 - 200
https://ammanvalley.foss.wales/t/4 - 404
https://ammanvalley.foss.wales/t/5 - 404
https://ammanvalley.foss.wales/t/6 - 404
https://ammanvalley.foss.wales/t/7 - 502  -- 502
https://ammanvalley.foss.wales/t/8 - 404
https://ammanvalley.foss.wales/t/9 - 404
https://ammanvalley.foss.wales/t/10 - 200
https://ammanvalley.foss.wales/t/11 - 200
https://ammanvalley.foss.wales/t/12 - 200
https://ammanvalley.foss.wales/t/13 - 200
https://ammanvalley.foss.wales/t/14 - 200
https://ammanvalley.foss.wales/t/15 - 200
https://ammanvalley.foss.wales/t/16 - 404
https://ammanvalley.foss.wales/t/17 - 404
https://ammanvalley.foss.wales/t/18 - 200
https://ammanvalley.foss.wales/t/19 - 200
https://ammanvalley.foss.wales/t/20 - 404
https://ammanvalley.foss.wales/t/21 - 200
https://ammanvalley.foss.wales/t/22 - 200
https://ammanvalley.foss.wales/t/23 - 200
https://ammanvalley.foss.wales/t/24 - 200
https://ammanvalley.foss.wales/t/25 - 200
https://ammanvalley.foss.wales/t/26 - 404
https://ammanvalley.foss.wales/t/27 - 404
https://ammanvalley.foss.wales/t/28 - 200
https://ammanvalley.foss.wales/t/29 - 404
https://ammanvalley.foss.wales/t/30 - 200
https://ammanvalley.foss.wales/t/31 - 200
https://ammanvalley.foss.wales/t/32 - 200
https://ammanvalley.foss.wales/t/33 - 200
https://ammanvalley.foss.wales/t/34 - 200
https://ammanvalley.foss.wales/t/35 - 200
https://ammanvalley.foss.wales/t/36 - 404
https://ammanvalley.foss.wales/t/37 - 200
https://ammanvalley.foss.wales/t/38 - 200
https://ammanvalley.foss.wales/t/39 - 200
https://ammanvalley.foss.wales/t/40 - 404
https://ammanvalley.foss.wales/t/41 - 200
https://ammanvalley.foss.wales/t/42 - 200
https://ammanvalley.foss.wales/t/43 - 200
https://ammanvalley.foss.wales/t/44 - 200
https://ammanvalley.foss.wales/t/45 - 200
https://ammanvalley.foss.wales/t/46 - 200
https://ammanvalley.foss.wales/t/47 - 404
https://ammanvalley.foss.wales/t/48 - 200
https://ammanvalley.foss.wales/t/49 - 200
https://ammanvalley.foss.wales/t/50 - 200
https://ammanvalley.foss.wales/t/51 - 200
https://ammanvalley.foss.wales/t/52 - 200
https://ammanvalley.foss.wales/t/53 - 200
https://ammanvalley.foss.wales/t/54 - 200
https://ammanvalley.foss.wales/t/55 - 200
https://ammanvalley.foss.wales/t/56 - 200
https://ammanvalley.foss.wales/t/57 - 200
https://ammanvalley.foss.wales/t/58 - 502 -- 502
https://ammanvalley.foss.wales/t/59 - 200
https://ammanvalley.foss.wales/t/60 - 200
https://ammanvalley.foss.wales/t/61 - 200
https://ammanvalley.foss.wales/t/62 - 404
https://ammanvalley.foss.wales/t/63 - 200
https://ammanvalley.foss.wales/t/64 - 404
https://ammanvalley.foss.wales/t/65 - 404
https://ammanvalley.foss.wales/t/66 - 200
https://ammanvalley.foss.wales/t/67 - 200
https://ammanvalley.foss.wales/t/68 - 404
https://ammanvalley.foss.wales/t/69 - 200
https://ammanvalley.foss.wales/t/70 - 404
https://ammanvalley.foss.wales/t/71 - 404
https://ammanvalley.foss.wales/t/72 - 404
https://ammanvalley.foss.wales/t/73 - 200
https://ammanvalley.foss.wales/t/74 - 404
https://ammanvalley.foss.wales/t/75 - 404
https://ammanvalley.foss.wales/t/76 - 404
https://ammanvalley.foss.wales/t/77 - 404
https://ammanvalley.foss.wales/t/78 - 404
https://ammanvalley.foss.wales/t/79 - 404
https://ammanvalley.foss.wales/t/80 - 404
https://ammanvalley.foss.wales/t/81 - 404
https://ammanvalley.foss.wales/t/82 - 404
https://ammanvalley.foss.wales/t/83 - 404
https://ammanvalley.foss.wales/t/84 - 404
https://ammanvalley.foss.wales/t/85 - 404
https://ammanvalley.foss.wales/t/86 - 404
https://ammanvalley.foss.wales/t/87 - 404
https://ammanvalley.foss.wales/t/88 - 404
https://ammanvalley.foss.wales/t/89 - 404
https://ammanvalley.foss.wales/t/90 - 404
https://ammanvalley.foss.wales/t/91 - 404
https://ammanvalley.foss.wales/t/92 - 404
https://ammanvalley.foss.wales/t/93 - 404
https://ammanvalley.foss.wales/t/94 - 404
https://ammanvalley.foss.wales/t/95 - 404
https://ammanvalley.foss.wales/t/96 - 404
https://ammanvalley.foss.wales/t/97 - 404
https://ammanvalley.foss.wales/t/98 - 404
https://ammanvalley.foss.wales/t/99 - 404
https://ammanvalley.foss.wales/t/100 - 404

Logs

A list of errors from ./launcher logs app:

...
[Wed 26 Oct 2022 04:28:27 PM UTC] Run reload cmd: sv reload nginx
fail: nginx: runsv not running
[Tue 25 Oct 2022 01:07:47 PM UTC] Run reload cmd: sv reload nginx
warning: nginx: unable to open supervise/ok: file does not exist
[Tue 25 Oct 2022 01:07:47 PM UTC] ESC[1;31mReload error for :ESC[0m
Started runsvdir, PID is 552
[Tue 25 Oct 2022 01:07:45 PM UTC] Run reload cmd: sv reload nginx
warning: nginx: unable to open supervise/ok: file does not exist
[Tue 25 Oct 2022 01:07:45 PM UTC] ESC[1;31mReload error for :ESC[0m
[Tue 25 Oct 2022 01:07:46 PM UTC] Domains not changed.

Further investigation

I am unsure about how to proceed with this. It affects only two topics.
The previews work, the data does not:
https://ammanvalley.foss.wales/t/how-this-forum-is-hosted/13/5?u=niccokunzmann

How do I restore the posts?

pfaffman · Outubro 26, 2022, 7:50pm

Esses tópicos foram excluídos? Eles poderiam ser MPs? Você os está solicitando como administrador?

Ah, desculpe, é um 502; você pode dar uma olhada em /var/discourse/shared/logs/rails/production.log e ver se há alguma pista lá. Veja também https://ammanvalley.foss.wales/logs

niccokunzmann · Outubro 26, 2022, 8:59pm

Hm, o rails está feliz e os logs são muito rápidos.

Started GET "/t/wip-arduino-workshop/58" for 8.8.8.8 at 2022-10-26 20:57:4
8 +0000
Processing by TopicsController#show as HTML
  Parameters: {"slug"=>"wip-arduino-workshop", "topic_id"=>"58"}
start
done

Então, o site ainda carrega por um tempo até que o 502 apareça.

No entanto, recebo muitos erros aqui /var/discourse/shared/standalone/log/rails/production.log:

Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 2 Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) subscribe failed, reconnecting in 1 second. Call stack /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:398:in `rescue in establish_connection'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:379:in `establish_connection'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:115:in `block in connect'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:344:in `with_reconnect'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:114:in `connect'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:417:in `ensure_connected'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:269:in `block in process'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:356:in `logging'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:268:in `process'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:175:in `block in call_loop'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:331:in `with_socket_timeout'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/client.rb:174:in `call_loop'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/subscribe.rb:44:in `subscription'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/subscribe.rb:14:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis.rb:287:in `_subscription'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/commands/pubsub.rb:20:in `block in subscribe'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis.rb:264:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis.rb:264:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis.rb:264:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.7.1/lib/redis/commands/pubsub.rb:19:in `subscribe'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-4.2.0/lib/message_bus/backends/redis.rb:302:in `global_subscribe'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-4.2.0/lib/message_bus.rb:763:in `global_subscribe_thread'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-4.2.0/lib/message_bus.rb:734:in `block in new_subscriber_thread' 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Error fetching job: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
 
Failed to report error: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL) 3 Job exception: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)
 
2022-10-26 16:31:02 +0000: Issuing stop to sidekiq
I, [2022-10-26T16:31:04.563256 #2079]  INFO -- : executing ["/var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn", "-E", "production", "-c", "config/unico\nrn.conf.rb", {10=>#<Unicorn::TCPSrv:fd 10, AF_INET, 127.0.0.1, 3000>}] (in /var/www/discourse)
I, [2022-10-26T16:31:04.563661 #2079]  INFO -- : forked child re-executing...
I, [2022-10-26T16:31:07.250986 #2079]  INFO -- : inherited addr=127.0.0.1:3000 fd=10
I, [2022-10-26T16:31:07.252783 #2079]  INFO -- : Refreshing Gem list
I, [2022-10-26T16:31:44.617034 #2079]  INFO -- : starting 1 supervised sidekiqs
I, [2022-10-26T16:31:47.752185 #2079]  INFO -- : master process ready
I, [2022-10-26T16:31:55.330063 #589]  INFO -- : reaped #<Process::Status: pid 704 exit 0> worker=1
I, [2022-10-26T16:31:55.333326 #589]  INFO -- : reaped #<Process::Status: pid 714 exit 0> worker=2
I, [2022-10-26T16:31:55.333928 #589]  INFO -- : reaped #<Process::Status: pid 1740 exit 0> worker=0
I, [2022-10-26T16:31:55.335869 #589]  INFO -- : master complete
I, [2022-10-26T16:32:26.504213 #2414]  INFO -- : worker=0 ready
2022-10-26 16:32:27 +0000: Issuing stop to sidekiq
I, [2022-10-26T16:32:28.194731 #2429]  INFO -- : worker=1 ready
I, [2022-10-26T16:32:29.025684 #2445]  INFO -- : worker=2 ready
Process would not terminate cleanly, force quitting. pid: 2402 Demon::Sidekiq
I, [2022-10-26T16:32:38.212437 #2079]  INFO -- : reaped #<Process::Status: pid 2429 exit 0> worker=1
I, [2022-10-26T16:32:38.212904 #2079]  INFO -- : reaped #<Process::Status: pid 2445 exit 0> worker=2
oxipng worker: could not get version of oxipng at /usr/local/bin/oxipng; please provide proper binary or disable this worker (--no-oxipng argument or `:oxipng => false` through options)
oxipng worker: could not get version of oxipng at /usr/local/bin/oxipng; please provide proper binary or disable this worker (--no-oxipng argument or `:oxipng => false` through options)
E, [2022-10-26T17:47:01.151902 #5094] ERROR -- : worker=0 PID:7920 timeout (31s > 30s), killing
I, [2022-10-26T21:01:44.288907 #10359]  INFO -- : worker=1 ready
I, [2022-10-26T21:02:06.344069 #10391]  INFO -- : worker=0 ready
I, [2022-10-26T21:02:10.336134 #10410]  INFO -- : worker=2 ready
D, [2022-10-26T21:02:14.022594 #595] DEBUG -- : waiting 16.0s after suspend/hibernation
E, [2022-10-26T21:02:39.047922 #595] ERROR -- : worker=1 PID:10359 timeout (31s > 30s), killing
E, [2022-10-26T21:02:39.175049 #595] ERROR -- : reaped #<Process::Status: pid 10359 SIGKILL (signal 9)> worker=1
E, [2022-10-26T21:02:44.202628 #595] ERROR -- : worker=0 PID:10391 timeout (31s > 30s), killing
E, [2022-10-26T21:02:44.327961 #595] ERROR -- : reaped #<Process::Status: pid 10391 SIGKILL (signal 9)> worker=0
I, [2022-10-26T21:03:08.803095 #10488]  INFO -- : worker=1 ready
E, [2022-10-26T21:03:11.378475 #595] ERROR -- : worker=2 PID:10410 timeout (31s > 30s), killing
E, [2022-10-26T21:03:11.456868 #595] ERROR -- : worker=2 PID:10410 timeout (31s > 30s), killing
E, [2022-10-26T21:03:11.548047 #595] ERROR -- : reaped #<Process::Status: pid 10410 SIGKILL (signal 9)> worker=2

Falco · Outubro 26, 2022, 9:03pm

Verifique também os arquivos em /var/log/nginx/ em busca de pistas.

pfaffman · Outubro 26, 2022, 9:04pm

Consigo ver que https://ammanvalley.foss.wales/t/welcome-to-amman-valley-makerspace/7.json não retorna nada e depois dá o 502.

Você tem algum plugin instalado?

Você reiniciou recentemente? Quanta RAM e swap você tem?

Parece que há algum problema com o Redis. Não consigo pensar em um motivo para que esses dois tópicos causem isso.

niccokunzmann · Outubro 27, 2022, 5:49am

https://ammanvalley.foss.wales/t/plugins-to-install/14

Não, mas reiniciei o aplicativo, o reconstruí e o atualizei através da interface web.

Então a saída do nginx espera o timeout em access.log e então produz isto:

[27/Oct/2022:05:45:48 +0000] "ammanvalley.foss.wales" 8.8.8.8 "GET /t/wip-arduino-workshop/58 HTTP/2.0" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:105.
0) Gecko/20100101 Firefox/105.0" "-" 502 217 "https://meta.discourse.org/" 30.516 30.516 "-" "-" "-" "-" "-" "-" "-"

No error.log, posso ver isto:

2022/10/26 21:03:48 [error] 578#578: *509 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 46.59.192.44, se\nrver: _, request: "GET /t/wip-arduino-workshop/58 HTTP/2.0", upstream: "http://127.0.0.1:3000/t/wip-arduino-workshop/58", host: "ammanvalley.foss.wales", refe\nrrer: "https://meta.discourse.org/"
2022/10/26 21:05:58 [error] 575#575: *591 upstream prematurely closed connection while reading response header from upstream, client: 189.79.134.158, server:
_, request: "GET /t/wip-arduino-workshop/58 HTTP/2.0", upstream: "http://127.0.0.1:3000/t/wip-arduino-workshop/58", host: "ammanvalley.foss.wales", referrer:
"https://meta.discourse.org/"
2022/10/26 21:52:07 [error] 578#578: *1117 upstream prematurely closed connection while reading response header from upstream, client: 216.68.248.7, server: _
, request: "GET /t/welcome-to-amman-valley-makerspace/7 HTTP/1.1", upstream: "http://127.0.0.1:3000/t/welcome-to-amman-valley-makerspace/7", host: "ammanvalle\ny.foss.wales"
2022/10/26 22:33:23 [error] 578#578: *1534 upstream prematurely closed connection while reading response header from upstream, client: 64.222.212.196, server:
 _, request: "GET /t/wip-arduino-workshop/58 HTTP/1.1", upstream: "http://127.0.0.1:3000/t/wip-arduino-workshop/58", host: "ammanvalley.foss.wales"
2022/10/27 05:21:21 [crit] 578#578: *2066 SSL_do_handshake() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handsh\naking, client: 64.62.197.155, server: 0.0.0.0:443
2022/10/27 05:44:54 [error] 578#578: *2073 upstream prematurely closed connection while reading response header from upstream, client: 92.40.170.157, server:
_, request: "GET /t/wip-arduino-workshop/58 HTTP/2.0", upstream: "http://127.0.0.1:3000/t/wip-arduino-workshop/58", host: "ammanvalley.foss.wales", referrer:
"https://meta.discourse.org/"
2022/10/27 05:45:48 [error] 578#578: *2073 upstream prematurely closed connection while reading response header from upstream, client: 92.40.170.157, server:
_, request: "GET /t/wip-arduino-workshop/58 HTTP/2.0", upstream: "http://127.0.0.1:3000/t/wip-arduino-workshop/58", host: "ammanvalley.foss.wales", referrer:
"https://meta.discourse.org/"
2022/10/27 05:48:48 [error] 575#575: *2161 upstream prematurely closed connection while reading response header from upstream, client: 92.40.170.156, server:
_, request: "GET /t/wip-arduino-workshop/58 HTTP/2.0", upstream: "http://127.0.0.1:3000/t/wip-arduino-workshop/58", host: "ammanvalley.foss.wales", referrer:
"https://meta.discourse.org/"

SWAP: 311M/2.97G RAM: 1.13G/1.93G

Benjamin_D · Outubro 27, 2022, 6:07am

o raw está lá, no entanto, e parece inofensivo.
Eu me pergunto o que o console do rails mostraria…

rails c
Topic.where(id: 58)

niccokunzmann · Outubro 28, 2022, 8:23am

Entrei no console do Rails.

pry(main)> Topic.where(id: 58)
=> [#<Topic:0x0000aaab01b22260
  id: 58,
  title: "[WIP] Arduino Workshop",
  last_posted_at: Sun, 23 Oct 2022 22:13:09.895053000 UTC +00:00,
  created_at: Fri, 23 Sep 2022 10:30:18.827248000 UTC +00:00,
  updated_at: Sun, 23 Oct 2022 22:13:10.291385000 UTC +00:00,
  views: 26,
  posts_count: 6,
  user_id: 1,
  last_post_user_id: 1,
  reply_count: 4,
  featured_user1_id: 2,
  featured_user2_id: nil,
  featured_user3_id: nil,
  deleted_at: nil,
  highest_post_number: 6,
  like_count: 1,
  incoming_link_count: 16,
  category_id: 6,
  visible: true,
  moderator_posts_count: 0,
  closed: false,
  archived: false,
  bumped_at: Sun, 23 Oct 2022 22:13:09.895053000 UTC +00:00,
  has_summary: false,
  archetype: "regular",
  featured_user4_id: nil,
  notify_moderators_count: 0,
  spam_count: 0,
  pinned_at: nil,
  score: 11.233333333333333,
  percent_rank: 1.0,
  subtype: nil,
  slug: "wip-arduino-workshop",
  deleted_by_id: nil,
  participant_count: 2,
  word_count: 184,
  excerpt:
   "I can run a workshop to introduce into programming arduinos. I post this here, so that we can organize \n\nOrganization\n\n\n make a poster @DomAman made one\n\n get material down from the community center\n\n make a list of what…",
  pinned_globally: false,
  pinned_until: nil,
  fancy_title: "[WIP] Arduino Workshop",
  highest_staff_post_number: 6,
  featured_link: nil,
  reviewable_score: 0.0,
  image_upload_id: nil,
  slow_mode_seconds: 0,
  bannered_until: nil,
  external_id: nil>]

RGJ · Outubro 30, 2022, 5:17pm

Parece que https://ammanvalley.foss.wales/raw/58 funciona bem.
Quando isso começou?

niccokunzmann · Novembro 5, 2022, 12:03am

Muito obrigado pelo link bruto! Isso ajuda a fundamentar os tópicos.

Um novo link está com erro 502 agora: https://ammanvalley.foss.wales/t/hovercraft-project/93

Hm. Eu fiz atualizações, desativei plugins. O novo link, no entanto, foi criado logo depois. Nenhuma atualização, nenhum desativamento, nada administrativo.

Não! Agora funciona! Hm. Eu não entendo.
Agora, parece que foi resolvido: Todas as três postagens são exibidas em HTML sem nenhum erro. Devo fechar isso agora? Se alguém estiver interessado em dar continuidade a isso, por favor, comente. Deixo o fechamento para depois.

niccokunzmann · Novembro 6, 2022, 5:42pm

Não! Não está resolvido. O fórum reiniciou há 30 minutos e o 502 aparece em outras postagens:

https://ammanvalley.foss.wales/t/hovercraft-project/93
https://ammanvalley.foss.wales/t/welcome-to-amman-valley-makerspace/7
https://ammanvalley.foss.wales/t/wip-arduino-workshop/58

O fórum geralmente carrega lentamente às vezes. Existe uma maneira de aumentar o tempo limite? Os links brutos funcionam!

Eu realmente não sei o que fazer com isso.

RGJ · Novembro 6, 2022, 6:48pm

Então o 502 realmente acontece por causa do timeout, acontece um pouco mais de 30 segundos.

O que está acontecendo no Sidekiq? Há uma fila grande?

pfaffman · Novembro 6, 2022, 6:53pm

Então é isso que eu abordaria.

Eu dobraria sua RAM ou obteria uma CPU mais rápida. Se você estiver em hospedagem compartilhada (como a maioria dos droplets da Digital Ocean, você pode ter um “vizinho barulhento” que está te atrasando).

Além disso, se você ainda não reiniciou seu sistema operacional, ele pode ter atualizado algumas bibliotecas que estão consumindo RAM.

Stephen · Novembro 6, 2022, 6:54pm

Você está realmente rodando em uma CPU arm, ou essa tag está errada?

niccokunzmann · Novembro 6, 2022, 9:31pm

Sim, é uma CPU ARM. Acidentalmente, investiguei mais a fundo e descobri que, quando abro uma página 502, vejo este processo disparando:

unicorn worker[1] -E production -c config/unicorn.conf.rb

99,0 é a porcentagem da CPU:

E isso também permanece ativo vários segundos após uma página 502 ser retornada ao navegador. O que ele está fazendo?

Como posso verificar isso?

Hm. Estes são os requisitos mínimos. Além disso, a RAM nunca dispara ou sequer parece se mover.

RGJ · Novembro 6, 2022, 9:50pm

Então, o processo do unicorn é o responsável por renderizar a página e entregá-la ao nginx. Após 30 segundos, o nginx desiste, mas o processo do unicorn não sabe disso, então ele continua renderizando a página.

A questão é: por que o unicorn demora tanto para renderizar certos tópicos?

Acho que sua RAM está boa.

Você pode tentar usar o mini-profiler para ver o que está acontecendo. Aqui está como fazer isso (está no contexto de um problema não relacionado).

Você pode verificar o Sidekiq indo para /sidekiq como administrador.

niccokunzmann · Novembro 6, 2022, 9:57pm

Parece que nada realmente está acontecendo para o sidekiq: o único pico de processo surge quando o navegador envia o GET.

Eu notei que isso acontece com tópicos de resposta. Isso também pode ser apenas porque eu sou mais propenso a visitá-los.

niccokunzmann · Novembro 9, 2022, 3:57pm

Estou com problemas para abri-lo. O atalho não funciona. Quando cliquei em uma postagem que gera um 502, ela abriu na página onde cliquei e depois desapareceu. No momento, não sei como abri-la. Posso executar algum JavaScript.

niccokunzmann · Novembro 15, 2022, 11:19pm

Minha pergunta no momento é: existe uma maneira de aumentar o tempo limite do nginx de, digamos, 30 segundos para cerca de dois minutos? Isso provavelmente resolveria o problema.

https://stackoverflow.com/questions/63801913/nginx-timeout-20s-causes-502-bad-gateway
templates/web.letsencrypt.ssl.template.yml - acessa a configuração do nginx
Module ngx_http_proxy_module

Minha conclusão é que posso escrever um template que adiciona um arquivo .conf a /etc/nginx/sites-enabled/ que define os tempos limite.

Além disso, notei que observo esse 502 com maior probabilidade quando uma resposta foi postada. Minha suposição é que o fórum precisa reconstruir a página e isso leva um tempo. Isso está correto? Eu me pergunto: posso acionar a reconstrução de uma página do Discourse manualmente?

niccokunzmann · Novembro 17, 2022, 6:27pm

Minha tentativa:

/var/discourse# cat templates/web.proxy_read_timeout.template.yml 
#
# Define o tempo limite para leituras do proxy nginx em sistemas lentos.
# Veja https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout
# Veja https://meta.discourse.org/t/502-bad-gateway-for-only-two-topics/242981/20?u=niccokunzmann
#
run:
  - file:
     path: "/etc/nginx/conf.d/proxy_read_timeout.conf"
     contents: |
       proxy_read_timeout 180s;

Em seguida, adicionei em containers/app.yml:

templates:
# ...
  - "templates/web.proxy_read_timeout.template.yml"

Em seguida, fiz

./launcher rebuild app

Por enquanto, não vejo esses erros aparecerem.

Tópico		Respostas	Visualizações
Bursts of 502 Service Unavailable, pointers to debug Installation	16	1597	8 de Junho de 2024
502 bad gateway with clean rebuild and no plugins except discourse docker Installation	5	371	14 de Fevereiro de 2024
502 Bad Gateway - nginx/1.14.0 (Ubuntu): Unable to find image locally + Error response from daemon Installation	11	9106	16 de Agosto de 2023
Moving posts returns 502 bad gateway Installation performance , server-resources	72	4742	12 de Janeiro de 2025
Nginx issue, when i am stop and starting the app getting "502 Bad Gateway" Installation	15	433	16 de Abril de 2024

502 Bad Gateway apenas para dois Tópicos

HEAD requests

GET requests

Logs

Further investigation

Tópicos relacionados