Merging two Discourse forums

OH! You’re back on the import_topics solution. I was talking about re-running the importer second importer. Several importers count on import_id being a string.

It’s not immediately clear which solution is easier/preferable. I suppose if the import_topics solution works, that would be great!

Does this mean if I re-run Topic and Category Export/Import it should now work?

Every case is a special unique snowflake. :snowflake: You worked on the migration for @omarfilip , so you would know better than me.

The group and user id’s were being converted to integers, so that could have been causing problems. Everything in the LookupContainer uses strings as it should though.

Only if you can get today’s code to use, but wait until the tests-passed branch has it.

5 Likes

I re-ran the export/import with Discourse (1612818).

The only apparent difference was that the importer reported which emails it was invalidating during the import. The result of the process was still the same - only 5 posts got imported, and post ownership is still mismatched.

Checking the status of the code - does this mean not everything made it into tests-passed?

It’s in test-passed. The Travis build was cancelled for some reason.

If IMPORT_SOURCE still only created 5 posts, then I don’t know what’s happening. At this point someone needs to compare the data (post id’s, user emails, etc.) in the exported data with what’s in the target db and figure out why there’s a collision (user_custom_fields, post_custom_fields).

1 Like

Does the category importer care if a name has a duplicate, but the username and email are unique?

In the forum that’s being exported/imported there are several that are like that.

No, it never looks at name. If you’re using the new IMPORT_SOURCE support, then email address is the only thing that it looks at. If there’s a duplicate username but different email address, then a number will be put on the end, like “omar1”.

5 Likes

Success! :tada:

With @pfaffman’s guidance, deleting import_id from both Discourses before export/import worked:

rails c
UserCustomField.where(name: 'import_id').delete_all
GroupCustomField.where(name: 'import_id').delete_all
CategoryCustomField.where(name: 'import_id').delete_all
PostCustomField.where(name: 'import_id').delete_all

This did not work:
discourse import_category /category-export-2017-10-25-225306.json

but this did:
bundle exec discourse import_category /category-export-2017-10-25-225306.json

Having to delete import_id from the Discourse that was converted with the phpBB importer prevents me from doing a leisurely initial import first and an incremental import later, but I can live with that.

Many thanks to @neil and @pfaffman!

5 Likes

I don’t follow. You didn’t try the IMPORT_SOURCE?

1 Like

No. I didn’t realize it was not part of the Category Export import reference in the original post?

I finally got to run the IMPORT_SOURCE and it worked perfectly. Users were matched properly, and an incremental import with Gerhard’s phpBB importer completed successfully after the category import.

:beers:

Thanks, @neil!

5 Likes

I want to merge a smaller discourse forum into a single category within my larger discourse forum. Both forums are native to discourse. I did not import anything from any other forum software. Both forums are running as separate stand-alone DO droplets (and are up-to-date). Naturally, I want all user logins, avatars, and data to transfer. Will the process laid out by @pfaffman work for this? Or, would a different process be more appropriate for putting the smaller forum into a single category?

1 Like

I think that you want to do what’s described above

3 Likes

Thanks for the clarification (and reassurance). I get a bit nervous before I trying things like this. Will report back once complete in case anyone else stumbles upon this at a later date.

Did you have success?

I did. The only challenge I encountered was the usernames and email accounts merging - which wasn’t perfect. It was probably an error on my part because it was more complicated than I anticipated. After a ton of work, and hard conversations with established members, we decided to leave the past behind and start over (we originally merged from vBulletin which went well). In retrospect it was the best decision. We use AWS for all of our media uploads now instead of keeping any content on site which has kept costs down significantly. Full disclosure…I made the mistake back in the days of BBS to operate a political forum and continue to do so (I may switch to hockey or something a bit less stressful). Stale political conversations and bickering weren’t really worth saving…since the banter and bickering never stops and getting to thousands of threads isn’t so tough.

I could see how someone buying an existing community, merging separate knowledge bases, or combining corporate communities of practice would absolutely need a flawless merge. For my purpose it wasn’t a big deal. Apologies to all for not providing an update. When things didn’t go as planned and I couldn’t figure it out, I didn’t want to be a pain.

5 Likes

It’s still good to hear what specific areas gave you trouble so that future readers know to pay particular attention to them.

Thanks for the update!

5 Likes

Yes, thanks for the belated update.

It is really useful to hear about edge cases such as politics because your hard-won learning is a useful consideration for making decisions about similar categories and topic types in our forums…

4 Likes

Not sure if it was mentioned in this topic but there is a newer guide on merging two forums that might be of interest to people reading this thread:

6 Likes