How to do forum migration properly?

(Áron Hamvas) #1


So a little background story:
I need to migrate forums from a legacy solution (CollabNet Teamforge) to Discourse. Couple of thousand users, maybe a hundred thousand posts in tens of thousands of topics, in a couple of hundred discussion forums (equivalent to Discourse categories). Since I have not found any existing solution, I am doing the development of the migration tool myself.

What I can do, basically, is retrieve every post via SOAP API and insert them into the Discourse DB directly. I have done that, but I can see that the user statistics is a mess (does not reflect reality), and user tagging does not seem to be working properly either.

I am assuming that creating a user is not very complicated, just put them in the proper table and that is it. But how about when I insert a post into a database? Which tables should I update to avoid the above mentioned issues?

(Rafael dos Santos Silva) #2

You should follow the conventions on the import scripts instead of inserting direct into the database.

(Áron Hamvas) #3

Thanks! I will check them out and see where they lead me. :slight_smile:

(Áron Hamvas) #4

Okay, it was my guess that it is a migration issue, but in the meantime, I realized that the statistics presented on the user profile page are correct, but the data on the Users page is definitely not good.

According to the stats on my profile page, I have 123 posts created and 780 posts read. On the Users page, I have 2 replies and 4 in the Read column. Any ideas what can cause this? How are these calculated? Probably not the same way.

(Jay Pfaffman) #5

Most importers don’t import statistics about how many messages have been read and such. The stats page data is generated by a script that runs periodically. You can see /sidekiq to kick off those processes automatically (if you’re on production or have manually started sidekiq on a development environment).

(Áron Hamvas) #6

I do not import statistics either. Just import posts. But it seems that the user stats are not updated properly. Thanks for the sidekiq hint though. Do you know which recurring job is responsible for the update? I am yet to find a Discourse sidekiq job description.