Lot of sidekiq jobs during data import

Support
1

Hello Guys,

We have a huge dataset(7 Lac user, 2.5 Lac threads, 5 million posts, 1.5 attachments, During import discourse created huge amount(5357085) of sidekiq jobs. It seems that we should keep running sidekiq during import time otherwise it would effect other functionalities. During import Discourse by default disable the outgoing email. I want to make sure again as we are going to import data for live system and don’t want to have suprises.

@neil, @Falco What do you suggests? Thanks

Regards,
Junaid

2

Generally you should turn off Sidekiq during import (it’ll be faster) and then turn it on when it’s done. It’ll do things like rebake every post (all 5 million of them), so it’ll take a while.

4 Likes