How to use a bulk import script (e.g., vbulletin.rb)

Dev
1

I’m doing what appears to be a vbulletin 4 import with some millions of posts. The server I’m doing it on is very slow (users were importing at about 50/minute when I gave up). Back of the envelope math suggests it’ll take 3 months.

So I’ll give bulk import script a chance again. I run normal import scripts in a production container. The same container that was merrily running the standard script can’t access the Discourse database. But the bulk importer wants to access /var/run/postgresql/.s.PGSQL.5432 rather than use the stuff that production and the regular import script use. So I add this to web_only.yml.

  - volume:
      host: /var/discourse/shared/data/postgres_run
      guest: /var/run/postgresql

And now I can access postgres.

But then, I can’t access mysql. The same credentials that work in the regular script and that work inside the container from mysql command line don’t work. I get an access denied. I added a puts to see that the credentials were all there was expected, and they are, but the code from the regular script:

    @client =
      Mysql2::Client.new(host: DB_HOST, username: DB_USER, password: DB_PW, database: DB_NAME)

works just fine, but this

      @client =
        Mysql2::Client.new(
          host: host,
          username: username,
          password: password,
          database: database,
          reconnect: true,
        )

Does not. I’ve checked, and

    puts "#{username}:#{password}@#{host}/#{database}"
    puts "PASSWORD: #{password.inspect}"

show everything as expected.

Do I need to run this in a development environment? (Why?)

I’m pretty stumped on how the mysql connection works in mysql client and the regular import script, but not in the bulk script. The code looks the same. I don’t see anything about mysql in the base.rb scripts.

Or maybe the machine is haunted.

1 Like