Hello, I am merging two forums together via the tutorial present here. I’m getting stuck on the command gunzip < /path/to/data/other-2018-08-02-134227-v2018xxx.tar.gz | psql -d copyme It returns me plenty of lines with import errors in unintelligible characters. Here are the steps performed: Discourse installation via official method on VPS. Import via GUI of the backup to merge (forumother) Backup of the latter (to have a backup with the latest version of discourse) Import of the backup of the destination forum (forumdest) Import of the backup of forumother via SFTP in the folder /var/discourse/shared/standalone/backups/default. Then I go to the above folder and run the command tar xvzf forumother.tar.gz (with the real name of the backup). Direction cd /var/discourse then ./launcher enter app sudo -u postgres psql and CREATE DATABASE "copyme" ENCODING = 'utf8'; and I quit with ``q It’s now that I’m stuck. The command gunzip < public/backups/default/forumother.tar.gz | psql -d copyme does not work as it is. The error is psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "root" does not exist I can see that this is a connection problem, but I don’t see how to solve it. I tried logging in as the postgres user su - postgres and then running the command gunzip ... but that’s when the command output is completely inconsistent. postgres@testforum-app:~$ gunzip < /var/www/discourse/public/backups/default/forum-francophon e-home-assistant-2022-04-24-120337-v20220407195246.tar.gz | psql -d copyme3 invalid command \{wB invalid command \֬IeY┴WPC<P%£6؜◆BP␌U@Q█5 ␋┼┴▒┌␋␍ ␌└└▒┼␍ \ ␋┼┴▒┌␋␍ ␌└└▒┼␍ \];*░Һ ␋┼┴▒┌␋␍ ␌└└▒┼␍ \FMݤ!≥"≠±↓QʏCMHȽ◆)&E/M6!◆┐ ␋┼┴▒┌␋␍ ␌└└▒┼␍ \↑ ERROR: ␋┼┴▒┌␋␍ ␉≤├␊ ␊─┤␊┼␌␊ ° ␊┼␌␍␋┼± "UTF8": █│␊° █│37 █│49 ERROR: ␋┼┴▒┌␋␍ ␉≤├␊ ␊─┤␊┼␌␊ ° ␊┼␌␍␋┼± "UTF8": █│␉␉ ␋┼┴▒┌␋␍ ␌└└▒┼␍ \лT4٩3┼← ␋┼┴▒┌␋␍ ␌└└▒┼␍ \^EC◆≥]B2Ɣ?8[N→^G9←@±4?WY°O͖↓HY┬4 ERROR: ␋┼┴▒┌␋␍ ␉≤├␊ ␊─┤␊┼␌␊ ° ␊┼␌␍␋┼± "UTF8": █│␉2 ... invalid command \UWXJ&qkW*SQy2漆&%>N≤OI2·^;├ ␋┼┴▒┌␋␍ ␌└└▒┼␍ \2d-ɸjrR*֬&^_'Ƶ褚N2HhIu S3 invalid command \VI invalid command \}Jݐ invalid command \BQNr|΅tU~#_k#D,WHfqtC invalid command \syJx% invalid command \Eq invalid command \-KmJ invalid command \ invalid command \/ly=}E~ԚYj>nO invalid command \SM!{Co~UH^x)*O}9vUǙk}LQ invalid command \eH<=gTRPR1}{~/t!wiwhӒCґJmCĥ=;O^p*- invalid command \gq{cr}N;. invalid command \2p3a-.C:m_Y@c<4/mvΩߵj]SޙU"O}$[ invalid command \) invalid command \ invalid command \VOtDIM2+͆Yq4R invalid command \WHElnHBDm%?PBFrOڙ}d" invalid command \PjsחPmqJ[B@$Wt*!&|Zz))Q6AJY%!/6=JRJBPþA)YTZ┘=!M^␉CO)→└␋9┤E▒5█E≤Pі T│ȐOȾB)W┌Ad:}1,kNDoO')G^W5:B␊I│7!N;─À@#┘ ┼┴▒┌␋␍ ␌└└▒┼␍ \6◆3 ␋┼┴▒┌␋␍ ␌└└▒┼␍ \2�Ψ)VD4π] ERROR: ␋┼┴▒┌␋␍ ␉≤├␊ ␊─┤␊┼␌␊ ° ␊┼␌␍␋┼± "UTF8": █│89 ├±␊@├␊├°┤└↑▒:·$ I tried to search on the forum, but I didn’t find anything on this. So I count on your help and thank you in advance. PS: My English is bad, so it’s possible that I’m not using the right words for a good search on the forum or I’m misinterpreting the tutorial quoted at the beginning of this post.

[image] Anthony: The error is psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "root" does not exist Probably, psql is trying to use your current system user (root) which doesn’t exist in Postgres. You may need to specify the user when running psql: psql -U postgres And then check this step: [image]Merge two Discourse sites into one If you’re running the import in an official Docker container (recommended), you will need to reset the postgres password to provide it to the script, otherwise you may encounter an error that the postgres user cannot access the database. To change the password: sudo -u postgres psql \password postgres (enter the new password) \q

Thanks for the answer, I’ll try tonight and get back to you to tell you.

Hello again, Let me start by thanking you for your help. My answer is a bit long, but it details the steps taken. Ben after several tries since the last message, I still have the impossibility to execute the command gunzip < blabla with always an authentication error. discourse@testforum-app:/var/www/discourse$ gunzip < public/backups/default/forum-francophone-home-assistant-2022-04-24-120337-v20220407195246.tar.gz | psql -U postgres -d copyme psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: Peer authentication failed for user "postgres" I give you the steps performed : I started from a fresh official installation. My two backups are made with the latest version of Discourse. I import and restore my “forumDest” backup. I upload the “ForumOther” backup to /var/discourse/shared/standalone/backups/default. I connect with SSH I go to `cd /var/discourse I enter the container with the command ./launcher enter app. I unzip the archive “ForumOther.tar.gz” located in /var/discourse/shared/standalone/backups/default * I reset the password root@testforum-app:/var/www/discourse/public/backups/default# ls dump.sql.gz forum-francophone-home-assistant-2022-04-24-120337-v20220407195246.tar.gz home-assistant-communaute-francophone-2022-04-24-111032-v20220304162250.tar.gz uploads Here is the result of the ls command forum-francophone-home-assistant-2022-04-24-120337-v20220407195246.tar.gz being ForumOther and home-assistant-communaute-francophone-2022-04-24-111032-v20220304162250.tar.gz being ForumDest I reset the postgres password with the following commands: sudo -u postgres psql \password postgres (enter the new password) \q This gives : root@testforum-app:/var/www/discourse/public/backups/default# sudo -u postgres psql psql (13.6 (Debian 13.6-1.pgdg110+1)) Type "help" for help. postgres=# \password postgres Enter new password for user "postgres": Enter it again: postgres=# \q Creation of the “copyme” database root@testforum-app:/var/www/discourse/public/backups/default# su postgres postgres@testforum-app:/var/www/discourse/public/backups/default$ psql psql (13.6 (Debian 13.6-1.pgdg110+1)) Type "help" for help. postgres=# CREATE DATABASE "copyme2" ENCODING = 'utf8'; CREATE DATABASE postgres=# \q postgres@testforum-app:/var/www/discourse/public/backups/default$ I run the command postgres@testforum-app:/var/www/discourse/public/backups/default$ gunzip < forum-francophone-home-assistant-2022-04-24-120337-v20220407195246.tar.gz | psql -d copyme or `gunzip < forum-francophone-home-assistant-2022-04-24-120337-v20220407195246.tar.gz | psql -U postgres -d copyme as mentioned. I can’t see the beginning of the process, but at the end it always puts me invalid command \P�jvsחP�m��q��J[B��@$��W�t*��!&|Zz)��)��Q�6�AJ�Y��%!/6=�JR��J��BPþ�A�)�YTZ�j��=! M��^bCO)+mi9��u�p�E��a��50��E�y��P��і��TxȐ��OȾ��B�)Wl��Ad��:��}1��,kNDo�O�'�)G��^W��5�� :��B�eIx7!N��;q�À@�#j��* invalid command \�6��`�3� invalid command \�2��󟯹�Ψ�)��V��D��4{]� ERROR: invalid byte sequence for encoding "UTF8": 0x89 postgres@testforum-app:/var/www/discourse/public/backups/default$ So I understand that there is a problem with the encoding? What I tried: Disassociating the commands: Unzipping the ForumOther backup gunzip < forum-francophone-home-assistant-2022-04-24-120337-v20220407195246.tar.gz which seems to extract something in incomprehensible language Enter on the DB psql -d copyme postgres@testforum-app:/var/www/discourse/public/backups/default$ psql -d copyme psql (13.6 (Debian 13.6-1.pgdg110+1)) Type "help" for help. copyme=# But once connected to the “copyme” base I don’t see what to do. I looked at the following for the IMPORT commands, but same thing, where should they be executed? How to do it? With which user? Let’s try step by step. If I succeed, I will put more precisely my procedure by detailing a little more than the original tutorial (which has the merit to explain the main principles and to exist). Thanks again for your time and help.

Hello to all, I have made good progress on my problem. Now I’m stuck just before the import script is executed. I run bundle install and then IMPORT=1 bundle install and at this point I get this error An error occurred while installing tiny_tds(2.1.5, and Bundler cannot continue I still run the full command IMPORT=1 DB_NAME=copyme DB_PASS=password SOURCE_BASE_URL= UPLOADS_PATH=/var/www/discourse/public/uploads/default bundle exec ruby script/bulk_import/discourse_merger.rb This time, it’s this error The git source https://github.com/nlalonde/ruby-bbcode-to-md is not yet checked out. Please run `bundle install` before trying to start your application Do you have an idea? Thanks

Hello, Another step forward, in the goal of merging two forums. I don’t have any dependency problem anymore, but I seem to have a right problem when launching the script. I launch the script with a local base on an official installation of discourse. The script command that I run and the return : discourse@testforum-app:/var/www/discourse$ IMPORT=1 DB_NAME=copyme DB_PASS=hacf2022 SOURCE_B exec ruby script/bulk_import/discourse_merger.rb Loading application... Traceback (most recent call last): 3: from script/bulk_import/discourse_merger.rb:817:in ` ' 2: from script/bulk_import/discourse_merger.rb:817:in `new' 1: from script/bulk_import/discourse_merger.rb:21:in `initialize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.2.4/lib/active_record/dynam e:Class (NoMethodError) Did you mean? connection_db_config connection_pool connection_class connection_class? connection_class= Thanks I also join you the launched commands (that could be useful for others). Upload other.forum.backup in /var/discourse/shared/standalone/backups/default and extract tar xvzf other.forum.backup.tar.gz cd `/var/discourse ./laucher enter app su postgres psql CREATE DATABASE "copyme" ENCODING = 'utf8'; \password postgres \q exit su discourse gunzip < public/backups/default/dump.sql.gz | psql -d copyme exit Install dependances nano, sqlite3 et mysql2 With root user apt update && apt upgrade apt-get install -y nano sqlite3 libsqlite3-dev mariadb-server libmariadb-dev gem install sqlite3 mysql2 ruby bbcode to md git clone https://github.com/nlalonde/ruby-bbcode-to-md.git cd ruby-bbcode-to-md gem build ruby-bbcode-to-md.gemspec gem install ruby-bbcode-to-md-*.gem Install Tiny_tds You need FreeTDS for install Tiny_tds. wget http://www.freetds.org/files/stable/freetds-1.3.10.tar.gz tar -xzf freetds-1.3.10.tar.gz cd freetds-1.3.10 ./configure --prefix=/usr/local --with-tdsver=7.3 make make install For tiny_tds gem install tiny_tds gem install ruby-bbcode-to-md Run script su discourse bundle config unset deployment IMPORT=1 bundle install IMPORT=1 DB_NAME=copyme DB_PASS=passpostgres SOURCE_BASE_URL=http://other.forum.fr UPLOADS_PATH=/var/www/discourse/public/uploads/default bundle exec ruby script/bulk_import/discourse_merger.rb

[image] Anthony: /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.2.4/lib/active_record/dynam e:Class (NoMethodError) Did you mean? connection_db_config It looks like you got unlucky with the timing of when you are doing this. 11 days ago there is a commit upgrading Discourse to using Rails 7. The import scripts use the ActiveRecord::Base.connection_config method which was removed with Rails 7.0.0rc1. @loic It looks like the import scripts might have been missed when doing the upgrade.

Oh thank you for this answer, It’s a great news, because I didn’t see where it could come from. I will wait for the update, except if I can update the merge script myself? Thanks

Just opened a PR, it should be merged soon :slight_smile: https://github.com/discourse/discourse/pull/16685

Hello, Thank you very much for the follow-up and the reactivity. As the migration of two forums is a long way full of pitfalls I am again confronted with a problem. I have just made the modification to the discourse_merge.rb file and then re-run the command. I now have a problem with an existing HASH. I understand that it is a duplicate problem, but of what nature? I already deleted a duplicate user, after trying to rename it for my trial, but now it’s this token problem (yet I think duplicates at the user level are handled?) Thanks again for your help. discourse@testforum-app:/var/www/discourse$ IMPORT=1 DB_NAME=copyme DB_PASS=password SOURCE_BASE_URL=http://other.forum.fr UPLOADS_PATH=/var/www/discourse/public/uploads/default bundle exec ruby script/bulk_import/discourse_merger.rb Loading application... Starting... Preloading I18n... Fixing highest post numbers... Loading imported group ids... Loading imported user ids... Loading imported category ids... Loading imported topic ids... Loading imported post ids... Loading groups indexes... Loading users indexes... Loading categories indexes... Loading topics indexes... Loading posts indexes... Loading post actions indexes... merging users... copying email_tokens... Traceback (most recent call last): 7: from script/bulk_import/discourse_merger.rb:817:in ` ' 6: from script/bulk_import/discourse_merger.rb:71:in `start' 5: from /var/www/discourse/script/bulk_import/base.rb:100:in `run' 4: from script/bulk_import/discourse_merger.rb:82:in `execute' 3: from script/bulk_import/discourse_merger.rb:145:in `copy_user_stuff' 2: from script/bulk_import/discourse_merger.rb:445:in `copy_model' 1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/pg-1.3.5/lib/pg/connection.rb:262:in `copy_data' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/pg-1.3.5/lib/pg/connection.rb:262:in `get_last_result': ERROR: duplicate key value violates unique constraint "index_email_tokens_on_token_hash" (PG::UniqueViolation) DETAIL: Key (token_hash)=(07165399523fcdce0679e45950f0260b61386a5c886e3e68fa6cf047d84d1caf) already exists. CONTEXT: COPY email_tokens, line 1

Hi. Here I came to the end of this migration despite several problems. The last problems came from users registered with different primary addresses on the two forums, but also from missing tables on one forum. I will come back to put the complete procedure here as soon as I have some time. Thanks again to @Constanza_Abarca for his help.

Problem for merge two forum

Product Support Self-hosting

loic (Loïc Guitaut) May 9, 2022, 8:54am 9

Just opened a PR, it should be merged soon
https://github.com/discourse/discourse/pull/16685

Topic		Replies	Views
Merging 2 discourse forums Product Support	15	2974	January 8, 2021
Merging two Discourse forums Migration	30	5045	January 22, 2020
Use an import script that requires MySQL Sysadmins how-to	51	13869	December 12, 2023
Merge two Discourse sites into one Migrating to Discourse how-to	5	9903	January 25, 2025
Migrate a bbPress WordPress plugin forum to Discourse Migrating to Discourse how-to	33	13640	January 10, 2024

Problem for merge two forum

Related topics