I’ve been doing import testing from another software with a large board (~10 million posts) and I’ve noticed that while the import process is running, I see another Ruby process that appears to be sidekiq stacking up tons of queued processes (sidekiq showing 100,000+ enqueued processes). Is there a better way to handle this? Is it possible (or recommended) to disable sidekiq while running a large import? If so, how would I do that?
2 Likes
You could turn off site setting discourse narrative bot enabled to stop creating that message for every new user, but most of those jobs need to happen. There are some topics here about increasing the number of sidekiq jobs to process the queue more quickly.
3 Likes
Ok, sounds good. I think adding another sidekiq worker could help. I will probably also disable that setting as you suggest, at least for the import, though that is just from the users. My main problem will be processing that needs to happen on the millions of posts.
I found this thread on how to adjust the number of sidekiq workers: Imported users not receiving verification emails due to massive Sidekiq backlog
Thanks!
2 Likes
Migrations don’t turn discobot off during the migration? That’s kind of shocking.
2 Likes
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.