How to use a bulk import script (e.g., vbulletin.rb)

Development

AI-generated summary

pfaffman is experiencing issues with the bulk import script vbulletin.rb for Discourse. The script is unable to connect to the MySQL database, despite using the same credentials that work in the regular import script and the MySQL command line. pfaffman has tried adding puts statements to verify the credentials and has checked the code, but the issue persists. The script can access the PostgreSQL database after adding a volume to web_only.yml, but the MySQL connection is denied. pfaffman is unsure if running the script in a development environment could resolve the issue and is seeking help to understand why the MySQL connection works in some cases but not in the bulk script.

1

I’m doing what appears to be a vbulletin 4 import with some millions of posts. The server I’m doing it on is very slow (users were importing at about 50/minute when I gave up). Back of the envelope math suggests it’ll take 3 months.

So I’ll give bulk import script a chance again. I run normal import scripts in a production container. The same container that was merrily running the standard script can’t access the Discourse database. But the bulk importer wants to access /var/run/postgresql/.s.PGSQL.5432 rather than use the stuff that production and the regular import script use. So I add this to web_only.yml.

  - volume:
      host: /var/discourse/shared/data/postgres_run
      guest: /var/run/postgresql

And now I can access postgres.

But then, I can’t access mysql. The same credentials that work in the regular script and that work inside the container from mysql command line don’t work. I get an access denied. I added a puts to see that the credentials were all there was expected, and they are, but the code from the regular script:

    @client =
      Mysql2::Client.new(host: DB_HOST, username: DB_USER, password: DB_PW, database: DB_NAME)

works just fine, but this

      @client =
        Mysql2::Client.new(
          host: host,
          username: username,
          password: password,
          database: database,
          reconnect: true,
        )

Does not. I’ve checked, and

    puts "#{username}:#{password}@#{host}/#{database}"
    puts "PASSWORD: #{password.inspect}"

show everything as expected.

Do I need to run this in a development environment? (Why?)

I’m pretty stumped on how the mysql connection works in mysql client and the regular import script, but not in the bulk script. The code looks the same. I don’t see anything about mysql in the base.rb scripts.

Or maybe the machine is haunted.

1 Like