How to use the bbpress import script - or any other import script with mysql dependency

import

(Carl Moebis) #24

With the latest update ver 2.0.0 beta10 I’m now getting more errors with the bbpress import script (still can’t figure out why images are not being pulled over, but instead referring to their old URLs). Error after the importing topics and posts:

importing private messages...
      120 / 120 (100.0%)  [156526 items/min]  /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/mysql2-0.5.1/lib/mysql2/client.rb:131:in `_query': Table 'db.wp_bp_messages_messages' doesn't exist (Mysql2::Error)
	from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/mysql2-0.5.1/lib/mysql2/client.rb:131:in `block in query'
	from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/mysql2-0.5.1/lib/mysql2/client.rb:130:in `handle_interrupt'
	from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/mysql2-0.5.1/lib/mysql2/client.rb:130:in `query'
	from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-1.0.0/lib/patches/db/mysql2.rb:22:in `query'
	from script/import_scripts/bbpress.rb:505:in `bbpress_query'
	from script/import_scripts/bbpress.rb:453:in `block in import_private_messages'
	from /var/www/discourse/script/import_scripts/base.rb:870:in `block in batches'
	from /var/www/discourse/script/import_scripts/base.rb:869:in `loop'
	from /var/www/discourse/script/import_scripts/base.rb:869:in `batches'
	from script/import_scripts/bbpress.rb:452:in `import_private_messages'
	from script/import_scripts/bbpress.rb:32:in `execute'
	from /var/www/discourse/script/import_scripts/base.rb:46:in `perform'
	from script/import_scripts/bbpress.rb:510:in `<main>'

(Steve) #26

EDIT: After deeper inspection I found that (at least in my setup) the file was called

  - "templates/import/mysql-dep.template.yml"

NOTE that the correct spelling is “dep”, not “dev”.

=================

Hello! Thanks for posting this. I tried but got this error:

cat: templates/import/mysql-dev.template.yml: No such file or directory

I unfortunately don’t know enough about Docker to proceed…What am I doing wrong? Thank you so much. Here is the full output:

root@forum /var/discourse # ./launcher rebuild import
Ensuring launcher is up to date
Fetching origin
remote: Counting objects: 16, done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 16 (delta 12), reused 7 (delta 6), pack-reused 0
Unpacking objects: 100% (16/16), done.
From https://github.com/discourse/discourse_docker
   2ad56e6..d032356  master     -> origin/master
Updating Launcher
Updating 2ad56e6..d032356
Fast-forward
 discourse-setup       | 2 +-
 image/base/Dockerfile | 5 +++--
 launcher              | 4 ++--
 3 files changed, 6 insertions(+), 5 deletions(-)
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
cat: templates/import/mysql-dev.template.yml: No such file or directory
cd /pups && git pull && /pups/bin/pups --stdin
/usr/bin/docker: invalid reference format: repository name must be lowercase.
See '/usr/bin/docker run --help'.
cat: cids/import_bootstrap.cid: No such file or directory
"docker rm" requires at least 1 argument.
See 'docker rm --help'.

Usage:  docker rm [OPTIONS] CONTAINER [CONTAINER...]

Remove one or more containers
rm: cannot remove 'cids/import_bootstrap.cid': No such file or directory
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
root@forum /var/discourse #support

(Caliph Khan (Caliph Khan)) #27

did you get it resolved? Im too having this error


(Carl Moebis) #28

No. we just manually re-uploaded all of the images.


(Caliph Khan (Caliph Khan)) #29

So I found out the reason for the issue we were having. The import script is breaking when it is trying to get wp_bb_message_message table, that’s happening because you would not have BuddyPress private messages module (which create these table) installed in your WordPress.

I did manage to pass that error by removing the code in bbpress.rb from line 414 to 502
This

 def import_private_messages
puts "", "importing private messages..."
last_post_id = -1
total_posts = bbpress_query("SELECT COUNT(*) count FROM #{BB_PRESS_PREFIX}bp_messages_messages").first["count"]

threads = {}

total_count = bbpress_query("SELECT COUNT(*) count FROM #{BB_PRESS_PREFIX}bp_messages_recipients").first["count"]
current_count = 0

batches(BATCH_SIZE) do |offset|
  rows = bbpress_query(<<-SQL
    SELECT thread_id, user_id
      FROM #{BB_PRESS_PREFIX}bp_messages_recipients
  ORDER BY id
     LIMIT #{BATCH_SIZE}
    OFFSET #{offset}
  SQL
  ).to_a

  break if rows.empty?

  rows.each do |row|
    current_count += 1
    print_status(current_count, total_count, get_start_time('private_messages'))

    threads[row['thread_id']] ||= {
      target_user_ids: [],
      imported_topic_id: nil
    }
    user_id = user_id_from_imported_user_id(row['user_id'])
    if user_id && !threads[row['thread_id']][:target_user_ids].include?(user_id)
      threads[row['thread_id']][:target_user_ids] << user_id
    end
  end
end

batches(BATCH_SIZE) do |offset|
  posts =  bbpress_query(<<-SQL
    SELECT id,
           thread_id,
           date_sent,
           sender_id,
           subject,
           message
      FROM wp_bp_messages_messages
     WHERE id > #{last_post_id}
  ORDER BY thread_id, date_sent
     LIMIT #{BATCH_SIZE}
  SQL
  ).to_a

  break if posts.empty?

  last_post_id = posts[-1]["id"].to_i

  create_posts(posts, total: total_posts, offset: offset) do |post|
    if tcf = TopicCustomField.where(name: 'bb_thread_id', value: post['thread_id']).first
      {
        id: "pm#{post['id']}",
        topic_id: threads[post['thread_id']][:imported_topic_id],
        user_id: user_id_from_imported_user_id(post['sender_id']) || find_user_by_import_id(post['sender_id'])&.id || -1,
        raw: post['message'],
        created_at: post['date_sent'],
      }
    else
      # First post of the thread
      {
        id: "pm#{post['id']}",
        archetype: Archetype.private_message,
        user_id: user_id_from_imported_user_id(post['sender_id']) || find_user_by_import_id(post['sender_id'])&.id || -1,
        title: post['subject'],
        raw: post['message'],
        created_at: post['date_sent'],
        target_usernames: User.where(id: threads[post['thread_id']][:target_user_ids]).pluck(:username),
        post_create_action: proc do |new_post|
          if topic = new_post.topic
            threads[post['thread_id']][:imported_topic_id] = topic.id
            TopicCustomField.create(topic_id: topic.id, name: 'bb_thread_id', value: post['thread_id'])
          else
            puts "Error in post_create_action! Can't find topic!"
          end
        end
      }
    end
  end
end
end

You can also get past it by simply installing buddypress and enabling its private message module.