Migrate a XenForo forum to Discourse

[Cyb3r]

Hey guys so after trying for a week or so I decided to import the database on docker app which I hoped it will succeed without any issues. TBH I got some issues at first but it turned to be all easy to solve.

Now I would recommend doing this only once before you deploy your production site, after importing everything take a Discourse backup, wipe everything out and install new Discourse instance then restore the imported db.

Ok let’s begin:

1. First lets backup your old database from your old server, sign in to your Discourse server then:

    ssh USER@XENFORO_SERVER_IP  # Use -p argument to specify custom port
    mysqldump -u XENFORO_DATABASE_USER -p XENFORO_DATABASE_NAME > xen_db.sql
    exit
   

2. After installing Discourse start the app:

    ./launcher start app
   

3. Now lets pull the database backup from xenforo server:

    scp USER@XENFORO_SERVER_IP:~/xen_db.sql /var/discourse/shared/standalone/xen_db.sql
   

   If you changed the ssh port use -P argument

    scp -P 2222 USER@XENFORO_SERVER_IP:~/xen_db.sql /var/discourse/shared/standalone/xen_db.sql 
   

4. Login to docker machine, install and start mysql:

    docker exec -it app bash
    sudo apt-get update
    sudo apt-get upgrade
   

   During the installation you will be asked to set a root password for mysql

    sudo apt-get install mysql-server mysql-client libmysqlclient-dev
    service mysql start
   

5. Login to mysql and create new database:

    mysql -u root -p
   

   Enter your mysql root password then

    create database xenforo_db;
    exit;
   

6. Import your xenforo database into the new one:

    mysql -u root -p xenforo_db < /shared/xen_db.sql
    # Enter your mysql root password and wait until it finish
   

7. Include the gem in discourse Gemfile:

    echo "gem 'mysql2'" >>Gemfile
   

8. Run bundle install:

    bundle install --no-deployment
   

9. Edit the importer script and change the following:

   vi /var/www/discourse/script/import_scripts/xenforo.rb
   XENFORO_DB = "xenforo_db"
   @client = Mysql2::Client.new(
     host: "localhost",
     username: "root",
     password: "Your mysql root password",
     database: XENFORO_DB
   )
   

10. You are set to go now, run the importer:

    RAILS_ENV=production bundle exec ruby script/import_scripts/xenforo.rb

11. If you get the error 'Peer authentication failed for user "discourse"':

12. Edit the file /etc/postgresql/9.5/main/pg_hba.conf

13. change all ‘peer’ to ‘trust’ and save the file

14. restart postgres server: service postgresql restart

Run the importer again and this time it should work without any problem, be patient this might take a while based on your database size.

Hope this helps someone.

[RoldanLT]

What are the list of XenForo Data covered on this import?
I hope to import this list:

XenForo Page Nodes

• Pages

XenForo

• Profile Fields
• Member Groups
• Members
• Status Updates / Profile Comments
• Status Comments
• Private Messages
• Private Message Replies
• Attachments
• Member Ranks

XenForo Forums

• Forums
• Topics
• Posts
• Attachments

[RoldanLT]

@Cyb3r

bundle install --no-deployment

Got this error:

# bundle install --no-deployment
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.
Your Gemfile has no gem server sources. If you need gems that are not already on your machine, add a line like this to your Gemfile:
source 'https://rubygems.org'
Could not find gem 'mysql2' in any of the gem sources listed in your Gemfile.

[RoldanLT]

Different Error now.

root@discourse-app:/var/www/discourse# RAILS_ENV=production bundle exec ruby script/import_scripts/xenforo.rb
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
Traceback (most recent call last):
	31: from script/import_scripts/xenforo.rb:3:in `<main>'
	30: from script/import_scripts/xenforo.rb:3:in `require'
	29: from /var/www/discourse/script/import_scripts/base.rb:20:in `<top (required)>'
	28: from /var/www/discourse/script/import_scripts/base.rb:544:in `<class:Base>'
	27: from /var/www/discourse/lib/discourse.rb:411:in `system_user'
	26: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `find_by'
	25: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `all?'
	24: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `each'
	23: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `block in find_by'
	22: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:336:in `columns_hash'
	21: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:456:in `load_schema'
	20: from /usr/local/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
	19: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:459:in `block in load_schema'
	18: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/attribute_decorators.rb:51:in `load_schema!'
	17: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/attributes.rb:234:in `load_schema!'
	16: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:466:in `load_schema!'
	15: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:90:in `connection'
	14: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
	13: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:1008:in `retrieve_connection'
	12: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:380:in `connection'
	11: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:521:in `checkout'
	10: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:793:in `acquire_connection'
	 9: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:832:in `try_to_checkout_new_connection'
	 8: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:853:in `checkout_new_connection'
	 7: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:809:in `new_connection'
	 6: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `postgresql_connection'
	 5: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `new'
	 4: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `initialize'
	 3: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:684:in `connect'
	 2: from /usr/local/lib/ruby/gems/2.5.0/gems/pg-1.0.0/lib/pg.rb:56:in `connect'
	 1: from /usr/local/lib/ruby/gems/2.5.0/gems/pg-1.0.0/lib/pg.rb:56:in `new'
/usr/local/lib/ruby/gems/2.5.0/gems/pg-1.0.0/lib/pg.rb:56:in `initialize': FATAL:  Peer authentication failed for user "discourse" (PG::ConnectionBad)

[RoldanLT]

Yay no idea how long this will take

[RoldanLT]

@Cyb3r

change all ‘peer’ to ‘trust’ and save the file

This is correct right?

[RoldanLT]

@Cyb3r

Include the gem in discourse Gemfile:

Guide needs an update I think.
Before running this, directory must be within /var/www/discourse/.
cd /var/www/discourse/

[R_X]

여러분께 질문드리고 싶은 게 있어요. 왜 Xenforo에서 Discourse로 옮기셨나요? 저는 일주일 전에 포럼을 시작했는데요, Xenforo 기반 사이트를 몇 개 보고 인상 깊었어요. 특히 포인트 시스템 같은 플러그인 때문에요. 이사를 갈 때의 단점이나 이유를 알고 싶었어요. 감사합니다.

[Jack51 (Jack)]

Digital Ocean 드롭렛에서 100만 개의 게시글이 있는 Xenforo → Discourse 마이그레이션을 방금 완료했습니다. 아래는 제가 명령어별로 사용했던 방법입니다(최근 다른 vBulletin 보드에서 수행한 작업과 매우 유사합니다).

가져오기(Import)를 위해 최소 4 vCPU/8GB를 권장합니다.

이 스레드에서 마이그레이션을 진행하는 데 도움을 주신 모든 분께 감사드립니다. 읽고 또 읽으며 정말 즐거운 며칠이었습니다.

1 - Digital Ocean Discourse 1클릭 드롭렛 설치

2 - SSH를 통해 프롬프트에 따라 Discourse 설치 완료

SSH 콘솔 열기
root
(루트 비밀번호)
(엔터)
(도메인).com
등

3 - SFTP 로그인

sftp root@XXX.XXX.XX.XX
y (확인 요청 시)
yes
(루트 비밀번호)
put db.sql /var/discourse/shared/standalone/db.sql

4 - 웹사이트에 로그인하여 관리자 계정 설정

5 - SSH 로그인 - 프로세스 시작

ssh root@XXX.XXX.XX.XX
cd /var/discourse
./launcher start app
docker exec -it app bash
sudo apt-get update
sudo apt-get upgrade
y

6 - MariaDB 설치 (mysql 대체재)

apt-get update && apt-get install mariadb-server-10.3 libmariadbd-dev
y

7 - MySQL 데이터베이스 설정

service mysql start
mysql -u root -p
password
create database import_db;
exit;

8 - 덤프 가져오기 → MySQL 데이터베이스 전송**

mysql -u root -p import_db < /shared/db.sql
password

9 - GEM 파일

echo "gem 'mysql2'" >>Gemfile
echo "gem 'mysql2', require: false" >> /var/www/discourse/Gemfile
echo "gem 'php_serialize', require: false" >> /var/www/discourse/Gemfile
cd /var/www/discourse
su discourse -c 'bundle install --no-deployment --without test --without development --path vendor/bundle'
(빨간색 텍스트 결과는 무시)

10 - 설치 스크립트 구성

vi /var/www/discourse/script/import_scripts/xenforo.rb

---DB 이름/비밀번호, 접두사 등 필요에 따라 텍스트 파일 편집---

(esc)
:wq

11 - 번들 구성

bundle config set path 'vendor/bundle'
bundle config set without 'development:test'
bundle config unset deployment
su discourse -c 'bundle install'

12 - MySQL 구성 (이전 단계에서 수행 가능할 수도 있음)

mysql --version
sudo mysql -u root -p
password
ALTER USER 'root'@'localhost' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
exit

13 - 설치 스크립트 실행

su discourse -c 'bundle exec ruby script/import_scripts/xenforo.rb'

[Timelord]

@Jack51

vi /var/www/discourse/script/import_scripts/xenforo.rb

현재 이 마이그레이션을 완료하는 작업을 진행 중인데, 첨부 파일 관련해서 오류가 발생하는 것 같습니다.

RAILS_ENV=production bundle exec ruby script/import_scripts/xenforo.rb
Loading existing groups...
Loading existing users...
Loading existing categories...
Loading existing posts...
Loading existing topics...

creating users
Skipping 407 already imported users

importing categories...
       12 / 12 (100.0%)  [344731 items/min]
creating topics and posts
        2 / 4554 (  0.0%)  [17724 items/min]  Traceback (most recent call last):
        18: from script/import_scripts/xenforo.rb:396:in `<main>'
        17: from /var/www/discourse/script/import_scripts/base.rb:47:in `perform'
        16: from script/import_scripts/xenforo.rb:32:in `execute'
        15: from script/import_scripts/xenforo.rb:174:in `import_posts'
        14: from /var/www/discourse/script/import_scripts/base.rb:866:in `batches'
        13: from /var/www/discourse/script/import_scripts/base.rb:866:in `loop'
        12: from /var/www/discourse/script/import_scripts/base.rb:867:in `block in batches'
        11: from script/import_scripts/xenforo.rb:180:in `block in import_posts'
        10: from /var/www/discourse/script/import_scripts/base.rb:508:in `create_posts'
         9: from /var/www/discourse/script/import_scripts/base.rb:508:in `each'
         8: from /var/www/discourse/script/import_scripts/base.rb:509:in `block in create_posts'
         7: from script/import_scripts/xenforo.rb:186:in `block (2 levels) in import_posts'
         6: from script/import_scripts/xenforo.rb:315:in `process_xenforo_post'
         5: from script/import_scripts/xenforo.rb:324:in `process_xf_attachments'
         4: from /usr/local/lib/ruby/2.7.0/set.rb:328:in `each'
         3: from /usr/local/lib/ruby/2.7.0/set.rb:328:in `each_key'
         2: from script/import_scripts/xenforo.rb:326:in `block in process_xf_attachments'
         1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.0.3.3/lib/active_support/core_ext/string/filters.rb:22:in `squish!'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.0.3.3/lib/active_support/core_ext/string/filters.rb:22:in `gsub!': can't modify frozen String: "\\t\\tSELECT a.attachment_id, a.data_id, d.fil\n\\t\\tFROM xf_attachment AS a\\n\\t\\tINNER JOIN xf_attachment_data d ON a.data_id = d.data_id\\n\\t\\tWHERE attachment_id = 13\\n" (FrozenError)


여기서 무슨 일이 일어나고 있는지 아는 분 계신가요? "FrozenError"는 무엇인가요?

[Timelord]

xenforo.rb 파일을 편집해서

# frozen_string_literal: true

를

# frozen_string_literal: false

으로 바꿔야 했던 것 같습니다.

[AstonJ]

이 가이드를 작성해 주셔서 감사합니다!

가져오기 도구가 아바타와 첨부 파일도 가져오나요? /tmp/attachments 폴더에 단순히 복사하면 되는 건가요?

[justin (Justin DiRose)]

가져오기를 간단히 살펴본 결과, 게시물 업로드만 가져오는 것으로 보입니다. 아바타 지원은 별도로 추가해야 합니다.

[Timelord]

확인했습니다. 업로드된 파일만 가져오며 아바타는 포함되지 않습니다. 지난 주에 Xenforo 마이그레이션을 완료했습니다.

[RoldanLT]

보드 크기는요?
게시물/스레드/멤버 수.
축하합니다