Notifications are slow causing major delays

Installation
1

I just had my instance merged without another one making the total amount of posts reach 1.5million with over 150000 registered users.

The issue is the notifications are delayed (likes received, private messages, new posts notifications, etcetera). This cause the Ai Bots (both discourse-ai and chatbot) to run replies after hours even days from the initial trigger (private message or mention).

Plenty of 504 errors as well when on /latest.json or in the admin panel trying to fetch reports.

I run a 3 core VPS with 4GB ram. I suspect the issue is cause of my server is overwhelmed. I’m also getting quite frequently the notice that the instance is showing as for unlogged in users due to large load.

  • Do i need a stronger server for a 1.5million post instance and 150000 reg users?

Mentioning i have average 100K daily consolidated page views (mostly crawlers) since my sitemap got updated with new content from the merge.

1 Like
2

You say you just merged with another instance, how recently was that? Or the real question behind that, if you visit /sidekiq while logged in as an admin, are there lots and lots of jobs in the queue?

Importing posts is a multi-step process with most of the processing delegated to jobs that get queued and run after the initial import.

It sounds like you may have imported a very large number of posts and potentially it’s just still catching up.

An issue I also had with an import (from a legacy system wrangled into generic email import, rather than Discourse merge) was that thousands of jobs were failing and continuously retrying which slowed down the queueing for everything else. I forget exactly what that was but I was able to fix the problem in the post content which stopped those jobs failing.

1 Like