Continuing the discussion from Importing from Vanilla: It took about 30 hours to import from Vanilla MySQL database. However, I have encountered a couple of issues that I hope someone could help me with. System reports a lot of queued jobs which stays constant at the same number - what does it even mean? The number of queued jobs is 1573616, which is high. This could indicate a problem with the Sidekiq process(es), or you may need to add more Sidekiq workers. Emails are not being sent - there are no connection attempts to the smtp server. I have another instance of Discourse running with exact same SMTP setup where it is working. SMTP testet ok on host via telnet. Can this be related to queued jobs? A lot of topics are without category - may be related to first problem?

[image] huulbaek: or you may need to add more Sidekiq workers. This is what you want to do - change the app.yml to launch more Sidekiq workers to process all the work from the import, until that’s all done.

How many topics / posts / categories / users are you importing? Topics that are missing categories are going to stay with missing categories, sounds like you have missing mapping somewhere. I would not expect any emails to come through before the job queue clears, may take quite a few hours.

[image] sam: How many topics / posts / categories / users are you importing? About 28.000 users - that was fast import, categories likewise. .I think it was 575.000 posts. Everything seems right in the MySQL database. [image] sam: I would not expect any emails to come through before the job queue clears, may take quite a few hours. The number is now 1.579.987 - so it seems to be going up if anything? :scream:

Problems after import from semi-large forum

Product Support

huulbaek (Thomas Huulbæk Titanium) October 9, 2015, 4:51pm 6

It eventually worked, increasing the workers. However, it still took more than 48 hours on a DigitalOcean 16GB droplet. But thanks!

Topic		Replies	Views
Lot of sidekiq jobs during data import Product Support	1	1694	November 25, 2016
Sidekiq During Import Product Support	3	949	April 29, 2020
Imported users not receiving verification emails due to massive Sidekiq backlog Product Support email	21	4950	February 21, 2016
Sidekiq queue keeps growing Product Support	15	3434	February 9, 2020
Large number of queued jobs Product Support	5	1692	July 3, 2017

Problems after import from semi-large forum

Related topics