Large number of queued jobs

Product Support

AI-generated summary

The discussion revolves around a large number of queued jobs in Sidekiq due to creating many topics in parallel via the API, which has slowed down the forum. leo.proctor is concerned about the impact of these queued jobs and asks about the specific Sidekiq jobs involved in post creation, as well as how to optimize performance. fefrei explains that asynchronous processing, such as notifications and image downloads, is normal and suggests increasing the number of Sidekiq workers to improve processing speed. leo.proctor then asks how to disable desktop notifications, which are appearing despite having disabled email notifications. fefrei clarifies that desktop notifications are sent immediately if users are watching the category and cannot be prevented after posts are created, but disabling outgoing mail should also stop queued notifications.

1

I’m creating huge number of topics in parallel via api, which has slowed down our forum incredibly. I’ve noticed several jobs are queued in sidekiq. Is this could be the reason? what sidekiq jobs are required for posts creation? fyi I’ve disabled sending emails.

Screen Shot 2017-07-03 at 4.02.54 PM.png1102×506 42.8 KB

I’m using discourse default settings for no. of sidekiq workers and database connections. Any advice for how much should I increase of any resource?

              total       used       free     shared    buffers     cached
Mem:          7983       6865       1118        278         60       3580
-/+ buffers/cache:       3224       4759
Swap:            0          0          0
2

This is normal – Discourse does some processing asynchronously, like sending notifications if necessary, or downloading remote images.

You can watch the progress by going to /sidekiq/queues?poll=true – you should see the queues and their (hopefully decreasing) sizes.

If you want to speed up this processing, consider raising the number of Sidekiq workers until allmost all RAM is used. :slight_smile:

3

Thanks for clarifying. Adding more sidekiq workers would be better.

sending notifications if necessary. How can I disable notifications and already queued notifications? I don’t want the users to be notified with bunch of notifications. I’ve already disabled emails by site settings.

4

If the posts have already been created, it’s already too late to properly manage notifications – your users will be notified as if these posts had been created normally (on mention, if mailing list mode is enabled, …).

If you only want to prevent emails, disabling all outgoing mail should also work for already queued notifications, I think.

5

I’ve disabled email notifications. I’m referring to desktop notifications, the one that appears on top right with avatar dropdown.

6

Users should only get them if they are watching the category, and these should have been sent out already because they are not delayed like emails.