I migrated my site to a new instance (Azure to Lightsail, 32gb ram) and am having a host of problems. The site is up and running but not seemingly working. It seems like I’m crashing my instance despite not having much traffic at the moment and having the max Lightsail instance. I can’t seem to do anything on the site: post threads, like posts, etc. I’m getting these errors: Message 'hijack topics timings ' is still running after 90 seconds on db default, this process may need to be restarted! Backtrace /var/www/discourse/lib/scheduler/defer.rb:105:in `block (2 levels) in do_work' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.8/lib/message_bus/timer_thread.rb:117:in `do_work' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/message_bus-4.3.8/lib/message_bus/timer_thread.rb:95:in `block (2 levels) in queue'``` [image] Failed to load resource: the server responded with a status of 503 () /admin/reports/bulk?reports%5Bdau_by_mau%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bdau_by_mau%5D%5Bstart_date%5D=2023-11-28&reports%5Bdau_by_mau%5D%5Bend_date%5D=2023-12-28&reports%5Bdaily_engaged_users%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bdaily_engaged_users%5D%5Bstart_date%5D=2023-11-28&reports%5Bdaily_engaged_users%5D%5Bend_date%5D=2023-12-28&reports%5Bnew_contributors%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bnew_contributors%5D%5Bstart_date%5D=2023-11-28&reports%5Bnew_contributors%5D%5Bend_date%5D=2023-12-28&reports%5Bpage_view_total_reqs%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bpage_view_total_reqs%5D%5Bstart_date%5D=2023-11-28&reports%5Bpage_view_total_reqs%5D%5Bend_date%5D=2023-12-28:1 Failed to load resource: the server responded with a status of 503 () /admin/reports/bulk?reports%5Bvisits%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bvisits%5D%5Bstart_date%5D=2023-11-28&reports%5Bvisits%5D%5Bend_date%5D=2023-12-28&reports%5Btime_to_first_response%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Btime_to_first_response%5D%5Bstart_date%5D=2023-11-28&reports%5Btime_to_first_response%5D%5Bend_date%5D=2023-12-28&reports%5Blikes%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Blikes%5D%5Bstart_date%5D=2023-11-28&reports%5Blikes%5D%5Bend_date%5D=2023-12-28&reports%5Bflags%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bflags%5D%5Bstart_date%5D=2023-11-28&reports%5Bflags%5D%5Bend_date%5D=2023-12-28:1 Failed to load resource: the server responded with a status of 503 () /admin/reports/bulk?reports%5Buser_to_user_private_messages_with_replies%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Buser_to_user_private_messages_with_replies%5D%5Bstart_date%5D=2023-11-28&reports%5Buser_to_user_private_messages_with_replies%5D%5Bend_date%5D=2023-12-28&reports%5Busers_by_type%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Busers_by_type%5D%5Bstart_date%5D=2023-12-28&reports%5Busers_by_type%5D%5Bend_date%5D=2023-12-28&reports%5Busers_by_trust_level%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Busers_by_trust_level%5D%5Bstart_date%5D=2023-12-28&reports%5Busers_by_trust_level%5D%5Bend_date%5D=2023-12-28&reports%5Bstorage_stats%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Bstorage_stats%5D%5Bstart_date%5D=2023-12-28&reports%5Bstorage_stats%5D%5Bend_date%5D=2023-12-28:1 Failed to load resource: the server responded with a status of 503 () /admin/reports/bulk?reports%5Btop_referred_topics%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Btop_referred_topics%5D%5Bstart_date%5D=2023-12-22&reports%5Btop_referred_topics%5D%5Bend_date%5D=2023-12-28&reports%5Btop_referred_topics%5D%5Blimit%5D=8&reports%5Btrending_search%5D%5Bfacets%5D%5B%5D=prev_period&reports%5Btrending_search%5D%5Bstart_date%5D=2023-11-28&reports%5Btrending_search%5D%5Bend_date%5D=2023-12-28&reports%5Btrending_search%5D%5Blimit%5D=8:1 Failed to load resource: the server responded with a status of 503 () 9953_2.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () 24.png:1 Failed to load resource: the server responded with a status of 503 () af975f5419770f3b2e0d38133d107299c2192045.png:1 Failed to load resource: the server responded with a status of 404 () posts:1 Failed to load resource: the server responded with a status of 502 () ajax-error.js:35 SyntaxError: Unexpected token ' ) at l (ajax-error.js:31:20) at u (ajax-error.js:74:10) at e.createPost (composer.js:1165:13) l @ ajax-error.js:35 u @ ajax-error.js:74 createPost @ composer.js:1165 presence.js:211 PresenceChannel '/discourse-presence/reply/14067' dropped message (received 28, expecting 29), resyncing... jquery.js:9940 POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable) send @ jquery.js:9940 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 flush @ screen-track.js:301 tick @ screen-track.js:365 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error (anonymous) @ screen-track.js:231 _ @ rsvp.js:434 y @ rsvp.js:420 m @ rsvp.js:369 p.invoke @ queue.ts:203 p.flush @ queue.ts:98 h.flush @ deferred-action-queues.ts:75 $._end @ index.ts:616 $.end @ index.ts:298 $._run @ index.ts:667 $.run @ index.ts:339 d @ index.js:108 t.error @ ajax.js:148 u @ jquery.js:3223 fireWith @ jquery.js:3353 x @ jquery.js:9629 (anonymous) @ jquery.js:9888 load (async) send @ jquery.js:9907 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 flush @ screen-track.js:301 tick @ screen-track.js:365 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 jquery.js:9940 POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable) send @ jquery.js:9940 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error (anonymous) @ screen-track.js:231 _ @ rsvp.js:434 y @ rsvp.js:420 m @ rsvp.js:369 p.invoke @ queue.ts:203 p.flush @ queue.ts:98 h.flush @ deferred-action-queues.ts:75 $._end @ index.ts:616 $.end @ index.ts:298 $._run @ index.ts:667 $.run @ index.ts:339 d @ index.js:108 t.error @ ajax.js:148 u @ jquery.js:3223 fireWith @ jquery.js:3353 x @ jquery.js:9629 (anonymous) @ jquery.js:9888 load (async) send @ jquery.js:9907 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 jquery.js:9940 POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable) send @ jquery.js:9940 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error (anonymous) @ screen-track.js:231 _ @ rsvp.js:434 y @ rsvp.js:420 m @ rsvp.js:369 p.invoke @ queue.ts:203 p.flush @ queue.ts:98 h.flush @ deferred-action-queues.ts:75 $._end @ index.ts:616 $.end @ index.ts:298 $._run @ index.ts:667 $.run @ index.ts:339 d @ index.js:108 t.error @ ajax.js:148 u @ jquery.js:3223 fireWith @ jquery.js:3353 x @ jquery.js:9629 (anonymous) @ jquery.js:9888 load (async) send @ jquery.js:9907 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 jquery.js:9940 POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable) send @ jquery.js:9940 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error (anonymous) @ screen-track.js:231 _ @ rsvp.js:434 y @ rsvp.js:420 m @ rsvp.js:369 p.invoke @ queue.ts:203 p.flush @ queue.ts:98 h.flush @ deferred-action-queues.ts:75 $._end @ index.ts:616 $.end @ index.ts:298 $._run @ index.ts:667 $.run @ index.ts:339 d @ index.js:108 t.error @ ajax.js:148 u @ jquery.js:3223 fireWith @ jquery.js:3353 x @ jquery.js:9629 (anonymous) @ jquery.js:9888 load (async) send @ jquery.js:9907 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 jquery.js:9940 POST https://forum.umhoops.com/topics/timings 503 (Service Unavailable) send @ jquery.js:9940 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 screen-track.js:231 Failed to update topic times for topic 14067 due to 503 error (anonymous) @ screen-track.js:231 _ @ rsvp.js:434 y @ rsvp.js:420 m @ rsvp.js:369 p.invoke @ queue.ts:203 p.flush @ queue.ts:98 h.flush @ deferred-action-queues.ts:75 $._end @ index.ts:616 $.end @ index.ts:298 $._run @ index.ts:667 $.run @ index.ts:339 d @ index.js:108 t.error @ ajax.js:148 u @ jquery.js:3223 fireWith @ jquery.js:3353 x @ jquery.js:9629 (anonymous) @ jquery.js:9888 load (async) send @ jquery.js:9907 ajax @ jquery.js:9521 f @ ajax.js:173 (anonymous) @ rsvp.js:459 R @ rsvp.js:915 k @ ajax.js:192 sendNextConsolidatedTiming @ screen-track.js:188 tick @ screen-track.js:370 (anonymous) @ screen-track.js:60 $._run @ index.ts:665 $.run @ index.ts:339 d @ index.js:108 (anonymous) @ screen-track.js:60 It was working at one point, then a few posts seem to have crashed the whole thing. Is it possible there is some sort of set of jobs running or something causing the issues?

[image] Falco: Did you tweak the app.yml to use that RAM? If that’s this setting, then yes. I doubled it to this. [image] [image] Falco: Did you run a database vacuum after the move? Is there a best practice of how to do this? I have not.

Ran VACUUM CLEAN in postgres. Still getting these errors in the log: 'hijack topics timings ’ is still running after 90 seconds on db default, this process may need to be restarted! ‘Track Visit’ is still running after 90 seconds on db default, this process may need to be restarted!

@dylanb I’m seeing this on an instance and was wondering if you got any further with your investigation?

503 errors, can't post, edit, like, etc

Product Support Self-hosting

Falco (Falco) December 28, 2023, 8:32pm 2

Sounds like a overwhelmed database.

Did you tweak the app.yml to use that RAM?

Did you run a database vacuum after the move?

Topic		Replies	Views
Moving posts returns 502 bad gateway Self-hosting performance , server-resources	71	6419	January 12, 2025
Users reporting lots of 502 errors when attempting to post due to "max consecutive replies" check Bug	36	3413	January 18, 2019
502 Bad Gateway for only two Topics Self-hosting arm	19	2311	November 17, 2022
User can't post: 500 internal server error Product Support	10	1631	April 18, 2018
Could sidekiq queue be reason for 500 errors? Self-hosting server-resources	30	4399	June 13, 2018

503 errors, can't post, edit, like, etc

Related topics