@gerhard thank you! I really appreciate that, it’s now running but I’m hitting an MySQL connection error. I’m hoping it’s just a credentials issue:
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/mysql2-0.5.1/lib/mysql2/client.rb:90:in `connect': Lost connection to MySQL server at 'reading initial communication packet', system error: 0 "Internal error/check (Not system error)" (Mysql2::Error::ConnectionError)That looks more like a can’t-connect-to-the-server issue than a credentials problem. Try entering the IP number for the server running (e.g., a docker container?) rather than localhost.
@pfaffman yeah I had the wrong port. The import is running now. In fact it’s been running for 26 hours now and it’s only at 45% completion. The database is around 570MB, 25,000 posts, 4,800 topics with 6,500 users. Is this time frame normal?
I’ve stopped the import a few times and rebooted the server, each time the import speeds up for a while then slows right down.
That’s too slow. I’d say it shouldn’t take more than 2-3 hours.
On what hardware are you running this? Is the database server on a different system with a slow connection? If so, it might be a better idea to create a database dump and use the MariaDB included in the import container.
Thanks again @gerhard clearly I’ve got an issue then! The database is coming from a 5 GB, 4 core SiteGround VPS to a 2 GB, 2 core DigitalOcean VPS.
Yeah. If you’re dragging the database from across the internet, it’ll be slow going.
@pfaffman it doesn’t seem to matter how I do this it’s slow going 
Its run for 11 more hours now using the MariaDB included in the import container and has only imported an additional 11%. So its now been running a total of 41 hours and completed 56%. The CPU on the 2 GB, 2 core DO server is maxed out. Any other thoughts on why this would be so slow? I’ve rebooted the server several times through this process and it speeds up for a while then drops to a crawl again.
It’s really hard to say what’s causing it. From the top of my head:
You caught a really shitty droplet. That happens sometimes. Creating a new one might help. Also, using a CPU Optimized Droplet is highly recommended for imports.
Your posts contain lots of images and/or attachments. I think that might cause a slow down.
And I’m out of ideas for now…
I’d really recommend running the import on your local machine, assuming that your PC is reasonable fast. With docker it’s quite easy and the performance is likely to be a lot better than with cloud servers. If it isn’t than there’s definitely something weird going on.
I have a last minute question guys. My forum is about to be launched.
I reorganized my forum categories after an import and I merged a lot of them.
However I have changed my mind and I’d like to “de-merge” some of them.
I think I could do something like:
Would that work? How do I delete my entire main category with all its topics and messages (5000+ topics)?
Deleting the topics doesn’t really delete them, so they won’t get re-imported.
I’m not quite clear what you’re trying to do, but I think your best bet is to just move those topics wherever you want them.
The incremental import doesn’t care what category the topics are in, so the new posts (has the forum remained active on the old site?) will be added to their respective topics.
Or maybe you want to just delete everything and run the whole import again.
Hi,
Thank you for your quick reply.
On phpbb I had a category A and a category B.
After my import on Discourse, I had a category A, a category B, then I moved all topics from A and B to a new category C, and I deleted my categories A and B.
I want to re-create my categories A and B (which is done in seconds), and get back my C topics in their original A and B categories. I can’t do this manually by selecting each topic one by one on hundred topics.
I don’t want to do this on my whole forum, it’s “just” hundreds of topics on several thousand.
Uh… 
Even by some rails command? 
Any risk removing posts from the database directly? The Discourse forum isn’t active yet, no user activity at all.
After having imported from phpbb, besides configuring stuff, no topics or replies were posted etc.
I also don’t want to do a full re-import because I spent so much time configuring stuff since my import.
@gerhard I cleaned up a lot of old irrelevant 2-years-ago posts in this topic. You may want to take a peek and see if anything was worth editing into the first post as additional info.
Here’s an idea. Leave the stuff in category C and set permissions so no one can see it, and/or delete them.
Then, in Rails, do
PostCustomField.where(name: 'import_id').destroy_all
That will delete the custom fields that let the importer know that the posts have been imported and it’ll import them again.
And just what “stuff” is that? If it’s just site settings, you can save them with GitHub - pfaffman/discourse-settings-uploader: Install a Discourse settings JSON file to a site. If it’s theme stuff, then it’s not too hard to export it and reload it.
Woah, thank you so much, I’m going to try that!
I’d be able to like you more than once, I’ll do it. 
Yes, it is site settings actually, plus some users permissions, changed usernames/emails, etc.
My import is fairly long, plus I had to do some custom commands in order to remove old bb tags (size and color), bulk move thousands of topics from categories to anothers and update the counts, etc…
Juste wondering about something. Does that mean that i’ll have about 5000 topics deleted but still in the database? What about the permalinks if I import again the topics with the same names?
Yes. They’ll be there. Yes, you’ll want to similarly delete the permalinks.
You could also delete the topics and posts from the rails console. If you want to delete all of them you could use a similar command. I’m not sure what other tables you might need to clean out.
I might give a try as it’s the simplest solution. Hoping that won’t break things…
Uh… I noticed something weird. My backups were about 300 MB before and I noticed that they are now more than 600 MB each.
I’m starting wondering if I didn’t remove some message through rails command a while ago and re-import them again or something like. I don’t remember doing such a thing, but I see no reason to have my backup size doubled… Is there any way to check the total number of messages (including deleted ones)? So I could compare with my phpbb forum number of messages.
edit: nevermind, I used data-explorer and I didn’t see any weird thing. Maybe it was because I had issues with imported images and I had to make a recovery from tombstone, which bringed back a lot of images (currently 300 MB of attached files).
Yeah I’ve created this topic to not be too off topic here.