Hello guys
We have 2 working discourse forums and we would like to merge them.
Is there a nice simple way to do this?
thanks
Not that I know of. I guess you’d have to write an importer. Should be pretty easy to do since the db schema is the same 
1 Like
@jmamazo did you do the merge the two discourse installations? Did you write an importer? I’m in the same situation…
When the script will be ready to be used in production @neil will write a topic for that.
5 Likes
Hello @neil
Now version 's Discourse is 2.1.0 beta 5. At this time, Everything is available for merge 2 Discourse forum?
Do we have step by step to merger them?
Thank you!
I have started a wiki post here:
4 Likes
Dear @neil. I have started this process but with little success.
It seems as though this discourse_merger is defaulting to using the user “discourse” to peer with the postgres instead of user “postgres”. I am getting inevitable peer authentication errors no matter the environment I work with.
A more in-depth explanation of exactly what environment/docker you are working with (and how exactly you set this up from a start-point) to successfully get this working would be much appreciated. I’m currently using most recent stable beta btw.
best,
Walker
I think there are some missing options for username, password in the initialize method:
https://github.com/discourse/discourse/blob/master/script/bulk_import/discourse_merger.rb#L18-L23
If you’re still trying to get this work, try adding user: and password: options for both of those connections and see if that fixes it.
2 Likes
Got it, will try. And thank you!!!
-Walker
I am going to try this and see.
def initialize
local_db = ActiveRecord::Base.connection_config
@raw_connection = PG.connect(dbname: local_db[:database], host: 'localhost', user: 'postgres', port: local_db[:port])
@source_db_config = {
dbname: ENV["DB_NAME"] || 'copyme',
host: ENV["DB_HOST"] || 'localhost',
user: ENV["DB_USER"] || 'postgres'
}It works with
@raw_connection = PG.connect(dbname: local_db[:database], host: 'localhost', user: 'postgres', password: '<put your password here>', port: local_db[:port])
It’s a hard-bake but at least it goes.
I had to do following to get password set:
sudo -u postgres psql postgres
\password postgres
(enter the new password)
\q
Now onto new and interesting bugs like unique constraints (instead of copy-over)!
best,
Walker
1 Like
Ok. So one (hopefully) more question.
I get this error right off the bat:
.rvm/gems/ruby-2.5.1/gems/pg-1.1.0/lib/pg/connection.rb:173:in `get_last_result': ERROR: duplicate key value violates unique constraint "index_topic_users_on_topic_id_and_user_id" (PG::UniqueViolation)
DETAIL: Key (topic_id, user_id)=(6233, 1213) already exists.
CONTEXT: COPY topic_users, line 1437
Does this mean that if I have a topic_id and user_id combo that is identical on both databases this will not work?
If so, that will become a problem in some use-cases where there is one or two admins that are replying a LOT topics on both forums that want to be merged. Because the assumption on this action is that it’s uni-directional and there are ideally no duplicate posts between forums (only duplicate user emails) can this script update the post ID with a unique # if it hits this error?
best,
Walker
I saw errors like that many many times while working on this and resolved all the times I saw it. Looks like there are more cases where it can happen.
No, it’s supposed to work. The users and topics will get new id’s after being copied, unless the user is matched to another user with the same email address.
Can you share more of the output above that error? Maybe I’ll be able to narrow it down.
1 Like
Ok. I reset the VM to just before the first command and hit it.
Ubuntu 16.04 Discourse Dev setup. Maybe a timing issue?
Here’s the full output:
ijm@ubuntu:~/discourse$ IMPORT=1 DB_NAME=copyme DB_USER=postgres SOURCE_BASE_URL=http://127.0.0.1 UPLOADS_PATH=/home/ijm/uploads/default bundle exec ruby script/bulk_import/discourse_merger.rb
Loading application...
Starting...
Preloading I18n...
Fixing highest post numbers...
Loading imported group ids...
Loading imported user ids...
Loading imported category ids...
Loading imported topic ids...
Loading imported post ids...
Loading groups indexes...
Loading users indexes...
Loading categories indexes...
Loading topics indexes...
Loading posts indexes...
Loading post actions indexes...
merging users...
copying email_tokens...
copying user_emails...
copying user_stats...
copying user_options...
copying user_profiles...
copying user_visits...
copying user_search_data...
copying given_daily_likes...
copying user_second_factors...
copying user_open_ids...
copying facebook_user_infos...
copying github_user_infos...
copying google_user_infos...
copying instagram_user_infos...
copying oauth2_user_infos...
copying single_sign_on_records...
copying twitter_user_infos...
copying email_change_requests...
copying groups...
copying group_users...
merging categories...
copying topics...
copying topic_allowed_groups...
copying topic_allowed_users...
copying topic_embeds...
copying topic_search_data...
copying topic_timers...
copying topic_users...
Traceback (most recent call last):
9: from script/bulk_import/discourse_merger.rb:799:in `<main>'
8: from script/bulk_import/discourse_merger.rb:67:in `start'
7: from /home/ijm/discourse/script/bulk_import/base.rb:81:in `run'
6: from script/bulk_import/discourse_merger.rb:81:in `execute'
5: from script/bulk_import/discourse_merger.rb:233:in `copy_topics'
4: from script/bulk_import/discourse_merger.rb:233:in `each'
3: from script/bulk_import/discourse_merger.rb:234:in `block in copy_topics'
2: from script/bulk_import/discourse_merger.rb:407:in `copy_model'
1: from /home/ijm/.rvm/gems/ruby-2.5.1/gems/pg-1.1.0/lib/pg/connection.rb:173:in `copy_data'
/home/ijm/.rvm/gems/ruby-2.5.1/gems/pg-1.1.0/lib/pg/connection.rb:173:in `get_last_result': ERROR: duplicate key value violates unique constraint "index_topic_users_on_topic_id_and_user_id" (PG::UniqueViolation)
DETAIL: Key (topic_id, user_id)=(6231, 1213) already exists.
CONTEXT: COPY topic_users, line 1437
2 Likes
No it shouldn’t be a timing issue.
I can’t see what needs to be changed when copying topic_users… Are you re-running the script on a database that has already had the tool run against it? Be sure that you are restoring the database back to before you ran the script. I guess you did that because you probably would have hit an error earlier… 
It’s very hard to debug this without some real data.
2 Likes
Most of this stuff has been answered now in the merger howto.
3 Likes