Importing Xenforo to Discourse

Cyb3r · November 21, 2016, 1:05am

I checked the importer script and it appears it doesn’t have an attachment importing code so that would need a custom modification to the script.

Are you thinking about moving your community to Discourse?

RoldanLT · November 21, 2016, 6:16am

Yes, XenForo somewhat bored me.

erlend_sh · November 21, 2016, 1:59pm

If you’d like to have the script also support

attachments
redirects (see vBulletin redirects for a point of reference)
passwords (see discourse-migratepassword – though I’d say this is lower priority since password reset works fine and is often better practice anyhow)

…we recommend posting to our #marketplace offering to pay a developer to work on these improvements. If @Cyb3r is not available for contracting work there’s sure to be others who’d be happy to take this on!

laogui · August 27, 2017, 2:23am

There’s an official XenForo importer, but unfortunately, there is no attachments import feature.

pfaffman · August 27, 2017, 12:53pm

If you’d like for it to support attachments,and have a budget, you can post in #marketplace.

0699392a91117d6baf72 · September 25, 2017, 6:56pm

hi, can u help me with converting xenforo to discourse? tnx

RoldanLT · July 14, 2018, 10:22pm

What are the list of XenForo Data covered on this import?
I hope to import this list:

XenForo Page Nodes

Pages

XenForo

Profile Fields
Member Groups
Members
Status Updates / Profile Comments
Status Comments
Private Messages
Private Message Replies
Attachments
Member Ranks

XenForo Forums

Forums
Topics
Posts
Attachments

RoldanLT · August 1, 2018, 12:05am

Got this error:

# bundle install --no-deployment
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.
Your Gemfile has no gem server sources. If you need gems that are not already on your machine, add a line like this to your Gemfile:
source 'https://rubygems.org'
Could not find gem 'mysql2' in any of the gem sources listed in your Gemfile.

RoldanLT · August 1, 2018, 1:08am

Different Error now.

root@discourse-app:/var/www/discourse# RAILS_ENV=production bundle exec ruby script/import_scripts/xenforo.rb
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
No connection to db, unable to retrieve site settings! (normal when running db:create)
Traceback (most recent call last):
	31: from script/import_scripts/xenforo.rb:3:in `<main>'
	30: from script/import_scripts/xenforo.rb:3:in `require'
	29: from /var/www/discourse/script/import_scripts/base.rb:20:in `<top (required)>'
	28: from /var/www/discourse/script/import_scripts/base.rb:544:in `<class:Base>'
	27: from /var/www/discourse/lib/discourse.rb:411:in `system_user'
	26: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `find_by'
	25: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `all?'
	24: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `each'
	23: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/core.rb:196:in `block in find_by'
	22: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:336:in `columns_hash'
	21: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:456:in `load_schema'
	20: from /usr/local/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
	19: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:459:in `block in load_schema'
	18: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/attribute_decorators.rb:51:in `load_schema!'
	17: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/attributes.rb:234:in `load_schema!'
	16: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/model_schema.rb:466:in `load_schema!'
	15: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:90:in `connection'
	14: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
	13: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:1008:in `retrieve_connection'
	12: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:380:in `connection'
	11: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:521:in `checkout'
	10: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:793:in `acquire_connection'
	 9: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:832:in `try_to_checkout_new_connection'
	 8: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:853:in `checkout_new_connection'
	 7: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:809:in `new_connection'
	 6: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `postgresql_connection'
	 5: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:40:in `new'
	 4: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:215:in `initialize'
	 3: from /usr/local/lib/ruby/gems/2.5.0/gems/activerecord-5.2.0/lib/active_record/connection_adapters/postgresql_adapter.rb:684:in `connect'
	 2: from /usr/local/lib/ruby/gems/2.5.0/gems/pg-1.0.0/lib/pg.rb:56:in `connect'
	 1: from /usr/local/lib/ruby/gems/2.5.0/gems/pg-1.0.0/lib/pg.rb:56:in `new'
/usr/local/lib/ruby/gems/2.5.0/gems/pg-1.0.0/lib/pg.rb:56:in `initialize': FATAL:  Peer authentication failed for user "discourse" (PG::ConnectionBad)

RoldanLT · August 1, 2018, 1:16am

Yay no idea how long this will take

RoldanLT · August 1, 2018, 2:46am

This is correct right?

RoldanLT · August 1, 2018, 2:55am

Guide needs an update I think.
Before running this, directory must be within /var/www/discourse/.
cd /var/www/discourse/

msinger · June 16, 2019, 12:23am

I used this script today and it worked well. The only issue I had was that it imported all of the banned users and soft deleted (hidden) posts/threads from Xenforo. That made hundreds of spam posts/threads visible in Discourse after the import.

If you have a lot of spam post/threads that you don’t want to import, then here are a few SQL changes you should consider making to the “xenforo.rb” script:

Add the WHERE clause below to ensure only active, unbanned users are imported.

def import_users
 puts '', "creating users"

 total_count = mysql_query("SELECT count(*) count FROM #{TABLE_PREFIX}user;").first['count']

 batches(BATCH_SIZE) do |offset|
   results = mysql_query(
     "SELECT user_id id, username, email, custom_title title, register_date created_at,
             last_activity last_visit_time, user_group_id, is_moderator, is_admin, is_staff
      FROM #{TABLE_PREFIX}user
      WHERE user_state = 'valid' AND is_banned = 0
      LIMIT #{BATCH_SIZE}
      OFFSET #{offset};")

2. Modify the WHERE clause below to ensure only visible threads/posts are imported.

def import_posts
 puts "", "creating topics and posts"

 total_count = mysql_query("SELECT count(*) count from #{TABLE_PREFIX}post").first["count"]

 posts_sql = "
     SELECT p.post_id id,
            t.thread_id topic_id,
            #{@prefix_as_category ? 't.prefix_id' : 't.node_id'} category_id,
            t.title title,
            t.first_post_id first_post_id,
            p.user_id user_id,
            p.message raw,
            p.post_date created_at
     FROM #{TABLE_PREFIX}post p,
          #{TABLE_PREFIX}thread t
     WHERE p.thread_id = t.thread_id AND p.message_state = 'visible' AND t.discussion_state = 'visible'
     ORDER BY p.post_date
     LIMIT #{BATCH_SIZE}" # needs OFFSET

codinghorror · June 16, 2019, 6:24am

Sure thanks for that, can we fold those changes into our script @techAPJ?

techAPJ · June 18, 2019, 10:24am

Done via:

Thanks for the tips @msinger.

codinghorror · July 17, 2019, 9:23am

8 posts were split to a new topic: Help migrating Xenforo

vladak · November 10, 2019, 10:29pm

Hi guy’s,

I sucssesufuly migrated Xenforo DB with 1500 memebr and ~75.000 posts to new servere where is Discourse.

One question about attachment… from script I can see that if attachments folder exist then script will do import, my question is where I need to put attac from old server in:

a) In Docker part of server (somewhere)
b) In origin location /var/discourse/shared/standalone/tmp/attachments

?

vladak · November 11, 2019, 5:30pm

Anyone know path???

pfaffman · November 11, 2019, 6:33pm

You put the files wherever you want that is accessible from the machine doing the import and put that path here:

  ATTACHMENT_DIR = '/tmp/attachments'

If you’re running inside a docker container, and put it under var/discourse/shared/standalone/tmp/attachments then you’d use /shared/tmp/attachments for the path. You can check by looking for the files when you’re inside the container.

vladak · November 11, 2019, 6:36pm

Okay Jay thank you for information, I’m just doing clean install this time with attachments folder setup, I will put then in /shared/tmp/attachments and get back here with results.

One quick question… is there anyway to import password for account’s too?