503 오류, 게시, 수정, 좋아요 등이 불가능

사이트를 새 인스턴스로 마이그레이션했습니다(Azure에서 Lightsail로, 32GB RAM) 그런데 여러 문제가 발생하고 있습니다.

사이트는 올라가 있긴 하지만 제대로 작동하지 않는 것 같습니다.

현재 트래픽이 많지 않고 Lightsail의 최대 인스턴스를 사용하고 있음에도 불구하고 인스턴스가 크래시 나는 것처럼 보입니다.

사이트에서 아무것도 할 수 없습니다: 스레드 게시, 게시물 좋아요 등.

다음과 같은 오류가 발생합니다:

Message

'hijack topics timings ' is still running after 90 seconds on db default, this process may need to be restarted!

Backtrace

/var/www/discourse/lib/scheduler/defer.rb:105:in `block (2 levels) in do_work'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.8/lib/message_bus/timer_thread.rb:117:in `do_work'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.8/lib/message_bus/timer_thread.rb:95:in `block (2 levels) in queue'```

Failed to load resource: the server responded with a status of 503 ()
/admin/reports/bulk?reports%5Bdau_by_mau%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bdau_by_mau%5D%5Bstart_date%5D=2023-11-28&reports%5Bdau_by_mau%5D%5Bend_date%5D=2023-12-28&reports%5Bdaily_engaged_users%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bdaily_engaged_users%5D%5Bstart_date%5D=2023-11-28&reports%5Bdaily_engaged_users%5D%5Bend_date%5D=2023-12-28&reports%5Bnew_contributors%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bnew_contributors%5D%5Bstart_date%5D=2023-11-28&reports%5Bnew_contributors%5D%5Bend_date%5D=2023-12-28&reports%5Bpage_view_total_reqs%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bpage_view_total_reqs%5D%5Bstart_date%5D=2023-11-28&reports%5Bpage_view_total_reqs%5D%5Bend_date%5D=2023-12-28:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
/admin/reports/bulk?reports%5Bvisits%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bvisits%5D%5Bstart_date%5D=2023-11-28&reports%5Bvisits%5D%5Bend_date%5D=2023-12-28&reports%5Btime_to_first_response%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Btime_to_first_response%5D%5Bstart_date%5D=2023-11-28&reports%5Btime_to_first_response%5D%5Bend_date%5D=2023-12-28&reports%5Blikes%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Blikes%5D%5Bstart_date%5D=2023-11-28&reports%5Blikes%5D%5Bend_date%5D=2023-12-28&reports%5Bflags%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bflags%5D%5Bstart_date%5D=2023-11-28&reports%5Bflags%5D%5Bend_date%5D=2023-12-28:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
/admin/reports/bulk?reports%5Buser_to_user_private_messages_with_replies%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Buser_to_user_private_messages_with_replies%5D%5Bstart_date%5D=2023-11-28&reports%5Buser_to_user_private_messages_with_replies%5D%5Bend_date%5D=2023-12-28&reports%5Busers_by_type%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Busers_by_type%5D%5Bstart_date%5D=2023-12-28&reports%5Busers_by_type%5D%5Bend_date%5D=2023-12-28&reports%5Busers_by_trust_level%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Busers_by_trust_level%5D%5Bstart_date%5D=2023-12-28&reports%5Busers_by_trust_level%5D%5Bend_date%5D=2023-12-28&reports%5Bstorage_stats%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bstorage_stats%5D%5Bstart_date%5D=2023-12-28&reports%5Bstorage_stats%5D%5Bend_date%5D=2023-12-28:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
/admin/reports/bulk?reports%5Btop_referred_topics%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Btop_referred_topics%5D%5Bstart_date%5D=2023-12-22&reports%5Btop_referred_topics%5D%5Bend_date%5D=2023-12-28&reports%5Btop_referred_topics%5D%5Blimit%5D=8&reports%5Btrending_search%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Btrending_search%5D%5Bstart_date%5D=2023-11-28&reports%5Btrending_search%5D%5Bend_date%5D=2023-12-28&reports%5Btrending_search%5D%5Blimit%5D=8:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
9953_2.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
24.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
24.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
24.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
24.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
24.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
24.png:1 
        
        
       Failed to load resource: the server responded with a status of 503 ()
af975f5419770f3b2e0d38133d107299c2192045.png:1 
        
        
       Failed to load resource: the server responded with a status of 404 ()
posts:1 
        
        
       Failed to load resource: the server responded with a status of 502 ()
ajax-error.js:35 SyntaxError: Unexpected token '<', "<html>
<h"... is not valid JSON
    at Function.parse [as parseJSON] (<anonymous>)
    at l (ajax-error.js:31:20)
    at u (ajax-error.js:74:10)
    at e.createPost (composer.js:1165:13)
l @ ajax-error.js:35
u @ ajax-error.js:74
createPost @ composer.js:1165
presence.js:211 PresenceChannel '/discourse-presence/reply/14067' dropped message (received 28, expecting 29), resyncing...
jquery.js:9940 
        
        
       POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable)
send @ jquery.js:9940
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
flush @ screen-track.js:301
tick @ screen-track.js:365
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error
(anonymous) @ screen-track.js:231
_ @ rsvp.js:434
y @ rsvp.js:420
m @ rsvp.js:369
p.invoke @ queue.ts:203
p.flush @ queue.ts:98
h.flush @ deferred-action-queues.ts:75
$._end @ index.ts:616
$.end @ index.ts:298
$._run @ index.ts:667
$.run @ index.ts:339
d @ index.js:108
t.error @ ajax.js:148
u @ jquery.js:3223
fireWith @ jquery.js:3353
x @ jquery.js:9629
(anonymous) @ jquery.js:9888
load (async)
send @ jquery.js:9907
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
flush @ screen-track.js:301
tick @ screen-track.js:365
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
jquery.js:9940 
        
        
       POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable)
send @ jquery.js:9940
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
tick @ screen-track.js:370
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error
(anonymous) @ screen-track.js:231
_ @ rsvp.js:434
y @ rsvp.js:420
m @ rsvp.js:369
p.invoke @ queue.ts:203
p.flush @ queue.ts:98
h.flush @ deferred-action-queues.ts:75
$._end @ index.ts:616
$.end @ index.ts:298
$._run @ index.ts:667
$.run @ index.ts:339
d @ index.js:108
t.error @ ajax.js:148
u @ jquery.js:3223
fireWith @ jquery.js:3353
x @ jquery.js:9629
(anonymous) @ jquery.js:9888
load (async)
send @ jquery.js:9907
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
tick @ screen-track.js:370
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
jquery.js:9940 
        
        
       POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable)
send @ jquery.js:9940
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
tick @ screen-track.js:370
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error
(anonymous) @ screen-track.js:231
_ @ rsvp.js:434
y @ rsvp.js:420
m @ rsvp.js:369
p.invoke @ queue.ts:203
p.flush @ queue.ts:98
h.flush @ deferred-action-queues.ts:75
$._end @ index.ts:616
$.end @ index.ts:298
$._run @ index.ts:667
$.run @ index.ts:339
d @ index.js:108
t.error @ ajax.js:148
u @ jquery.js:3223
fireWith @ jquery.js:3353
x @ jquery.js:9629
(anonymous) @ jquery.js:9888
load (async)
send @ jquery.js:9907
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
tick @ screen-track.js:370
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
jquery.js:9940 
        
        
       POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable)
send @ jquery.js:9940
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
tick @ screen-track.js:370
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60
screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error
(anonymous) @ screen-track.js:231
_ @ rsvp.js:434
y @ rsvp.js:420
m @ rsvp.js:369
p.invoke @ queue.ts:203
p.flush @ queue.ts:98
h.flush @ deferred-action-queues.ts:75
$._end @ index.ts:616
$.end @ index.ts:298
$._run @ index.ts:667
$.run @ index.ts:339
d @ index.js:108
t.error @ ajax.js:148
u @ jquery.js:3223
fireWith @ jquery.js:3353
x @ jquery.js:9629
(anonymous) @ jquery.js:9888
load (async)
send @ jquery.js:9907
ajax @ jquery.js:9521
f @ ajax.js:173
(anonymous) @ rsvp.js:459
R @ rsvp.js:915
k @ ajax.js:192
sendNextConsolidatedTiming @ screen-track.js:188
tick @ screen-track.js:370
(anonymous) @ screen-track.js:60
$._run @ index.ts:665
$.run @ index.ts:339
d @ index.js:108
(anonymous) @ screen-track.js:60

어느 시점에는 작동하다가, 몇몇 게시글이 올라간 후 전체가 크래시 난 것 같습니다. 문제를 일으키는 작업(job) 세트나 다른 것이 실행되고 있을 가능성이 있을까요?

데이터베이스가 과부하에 걸린 것 같습니다.

app.yml을 조정해서 그 RAM을 사용하도록 했나요?

이동 후 데이터베이스 vacuum을 실행했나요?

해당 설정이라면, 네. 이 값으로 2배로 늘렸습니다.

image

이를 수행하는 모범 사례가 있나요? 저는 아직 수행하지 않았습니다.

postgres에서 VACUUM CLEAN을 실행했습니다.

로그에 여전히 이러한 오류가 표시됩니다:
'hijack topics timings’가 db default에서 90초 후에도 여전히 실행 중입니다. 이 프로세스를 다시 시작해야 할 수 있습니다!
'Track Visit’가 db default에서 90초 후에도 여전히 실행 중입니다. 이 프로세스를 다시 시작해야 할 수 있습니다!

@dylanb 인스턴스에서 이 문제가 발생하고 있는데, 조사가 더 진전된 게 있는지 궁금합니다.