@gerhard thank you! I really appreciate that, it’s now running but I’m hitting an MySQL connection error. I’m hoping it’s just a credentials issue:
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/mysql2-0.5.1/lib/mysql2/client.rb:90:in `connect': Lost connection to MySQL server at 'reading initial communication packet', system error: 0 "Internal error/check (Not system error)" (Mysql2::Error::ConnectionError)
@pfaffman yeah I had the wrong port. The import is running now. In fact it’s been running for 26 hours now and it’s only at 45% completion. The database is around 570MB, 25,000 posts, 4,800 topics with 6,500 users. Is this time frame normal?
I’ve stopped the import a few times and rebooted the server, each time the import speeds up for a while then slows right down.
That’s too slow. I’d say it shouldn’t take more than 2-3 hours.
On what hardware are you running this? Is the database server on a different system with a slow connection? If so, it might be a better idea to create a database dump and use the MariaDB included in the import container.
@pfaffman it doesn’t seem to matter how I do this it’s slow going Its run for 11 more hours now using the MariaDB included in the import container and has only imported an additional 11%. So its now been running a total of 41 hours and completed 56%. The CPU on the 2 GB, 2 core DO server is maxed out. Any other thoughts on why this would be so slow? I’ve rebooted the server several times through this process and it speeds up for a while then drops to a crawl again.
It’s really hard to say what’s causing it. From the top of my head:
You caught a really shitty droplet. That happens sometimes. Creating a new one might help. Also, using a CPU Optimized Droplet is highly recommended for imports.
Your posts contain lots of images and/or attachments. I think that might cause a slow down.
And I’m out of ideas for now…
I’d really recommend running the import on your local machine, assuming that your PC is reasonable fast. With docker it’s quite easy and the performance is likely to be a lot better than with cloud servers. If it isn’t than there’s definitely something weird going on.
I have a last minute question guys. My forum is about to be launched.
I reorganized my forum categories after an import and I merged a lot of them.
However I have changed my mind and I’d like to “de-merge” some of them.
I think I could do something like:
Remove all the message from the category which a lot of messages from other categories were merged in
Import again my phpbb dump, so as it supports incremential import, it should re-import only all the message that were deleted, each in their own category
Would that work? How do I delete my entire main category with all its topics and messages (5000+ topics)?
On phpbb I had a category A and a category B.
After my import on Discourse, I had a category A, a category B, then I moved all topics from A and B to a new category C, and I deleted my categories A and B.
I want to re-create my categories A and B (which is done in seconds), and get back my C topics in their original A and B categories. I can’t do this manually by selecting each topic one by one on hundred topics.
I don’t want to do this on my whole forum, it’s “just” hundreds of topics on several thousand.
Uh… Even by some rails command?
Any risk removing posts from the database directly? The Discourse forum isn’t active yet, no user activity at all.
After having imported from phpbb, besides configuring stuff, no topics or replies were posted etc.
I also don’t want to do a full re-import because I spent so much time configuring stuff since my import.
Woah, thank you so much, I’m going to try that!
I’d be able to like you more than once, I’ll do it.
Yes, it is site settings actually, plus some users permissions, changed usernames/emails, etc.
My import is fairly long, plus I had to do some custom commands in order to remove old bb tags (size and color), bulk move thousands of topics from categories to anothers and update the counts, etc…
Uh… I noticed something weird. My backups were about 300 MB before and I noticed that they are now more than 600 MB each.
I’m starting wondering if I didn’t remove some message through rails command a while ago and re-import them again or something like. I don’t remember doing such a thing, but I see no reason to have my backup size doubled… Is there any way to check the total number of messages (including deleted ones)? So I could compare with my phpbb forum number of messages.
edit: nevermind, I used data-explorer and I didn’t see any weird thing. Maybe it was because I had issues with imported images and I had to make a recovery from tombstone, which bringed back a lot of images (currently 300 MB of attached files).