Migrating vBulletin 5 database - Import script errors

Crius
Tue, 20 Dec 2022 17:22:51 +0000
Migration

Ok so, quick recap.

I’m volunteering from the migration of a forum that is currently on vbulletin3.
On a staging environment, starting from a dump of the database (20GB, you read that right).

Run the upgrade to vBulletin 5. Took 5-6 hours but went through. Version is vBulletin 5.4.
Did some cleanup of the usernames to be accepted by discourse.

Now, installed the docker discourse and followed loosely this guide for the preparation. Loosely meaning that most of it was redundant or out of date but it helped getting some bearing about what to do.

I’m at the step in which I literally am going blind as I have near zero Ruby coding experience.
So, the relevant parts, after finishing the installation I’ve entered the container with ./launcher enter app then:

Added freetds-dev and libmariadb-dev

Edited the Gemfile to add php_serialize gem.

from shell, run export IMPORT=1 to set the environment for import

as discourse user run bundle install --no-deployment --without test --without development --path vendor/bundle

Got the error:

You are trying to install in deployment mode after changing
your Gemfile. Run `bundle install` elsewhere and add the
updated Gemfile.lock to version control.

If this is a development machine, remove the /var/www/discourse/Gemfile freeze
by running `bundle config unset deployment`.

The list of sources changed
The dependencies in your gemfile changed

You have added to the Gemfile:
* mysql2
* redcarpet
* php_serialize
* sqlite3 (~> 1.3, >= 1.3.13)
* ruby-bbcode-to-md
* reverse_markdown
* tiny_tds
* csv
* parallel

So, continue with

bundle config unset deployment and run again the previous command

checked that both mysql2 and php_serialize were there (they were)

added the old forum avatars (no attachments to import) and assigned the directories ownership to discourse user in its own /home/discourse

edited script/import_scripts/vbulletin5.rb to change the reference for connecting to the db

as user discourse run bundle exec ruby script/import_scripts/vbulletin5.rb

This returned to me an error about tzinfo Integer values not supported that I found being mentioned here on this discourse.

Loading existing groups...
Loading existing users...
Loading existing categories...
Loading existing posts...
Loading existing topics...

importing groups...
       41 / 41 (100.0%)  [2294 items/min]  ]
importing users
Traceback (most recent call last):
15: from script/import_scripts/vbulletin5.rb:726:in ` '
        14: from /var/www/discourse/script/import_scripts/base.rb:47:in `perform'
        13: from script/import_scripts/vbulletin5.rb:46:in `execute'
        12: from script/import_scripts/vbulletin5.rb:79:in `import_users'
        11: from /var/www/discourse/script/import_scripts/base.rb:916:in `batches'
        10: from /var/www/discourse/script/import_scripts/base.rb:916:in `loop'
         9: from /var/www/discourse/script/import_scripts/base.rb:917:in `block in batches'
         8: from script/import_scripts/vbulletin5.rb:98:in `block in import_users'
         7: from /var/www/discourse/script/import_scripts/base.rb:264:in `create_users'
         6: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/mysql2/alias_method.rb:8:in `each'
         5: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/mysql2/alias_method.rb:8:in `each'
         4: from /var/www/discourse/script/import_scripts/base.rb:265:in `block in create_users'
         3: from script/import_scripts/vbulletin5.rb:110:in `block (2 levels) in import_users'
         2: from script/import_scripts/vbulletin5.rb:718:in `parse_timestamp'
         1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/tzinfo-2.0.5/lib/tzinfo/timezone.rb:575:in `utc_to_local'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/tzinfo-2.0.5/lib/tzinfo/timestamp.rb:138:in `for': Integer values are not supported (ArgumentError)

@Haddoq suggestion was to change a line from Time.zone.at(@tz.utc_to_local(timestamp)) into Time.zone.at(timestamp).

It also suggest to add lastvisit in the user query because it will cause another error otherwise so I did that as well.

However, now when I launch the migration with bundle exec ruby script/import_scripts/vbulletin5.rb this is what I get:

Loading existing groups...
Loading existing users...
Loading existing categories...
Loading existing posts...
Loading existing topics...

importing groups...
       41 / 41 (100.0%)  [120217 items/min]
importing users
Traceback (most recent call last):
13: from script/import_scripts/vbulletin5.rb:727:in ` '
        12: from /var/www/discourse/script/import_scripts/base.rb:47:in `perform'
        11: from script/import_scripts/vbulletin5.rb:46:in `execute'
        10: from script/import_scripts/vbulletin5.rb:79:in `import_users'
         9: from /var/www/discourse/script/import_scripts/base.rb:916:in `batches'
         8: from /var/www/discourse/script/import_scripts/base.rb:916:in `loop'
         7: from /var/www/discourse/script/import_scripts/base.rb:917:in `block in batches'
         6: from script/import_scripts/vbulletin5.rb:80:in `block in import_users'
         5: from script/import_scripts/vbulletin5.rb:723:in `mysql_query'
         4: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/mysql2/alias_method.rb:22:in `query'
         3: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/mysql2-0.5.4/lib/mysql2/client.rb:147:in `query'
         2: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/mysql2-0.5.4/lib/mysql2/client.rb:147:in `handle_interrupt'
         1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/mysql2-0.5.4/lib/mysql2/client.rb:148:in `block in query'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/mysql2-0.5.4/lib/mysql2/client.rb:148:in `_query': You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CASE WHEN u.scheme='blowfish:10' THEN token (Mysql2::Error)
                 WHEN u.scheme='lega' at line 2

At this point I’m a little lost. Can someone help?

47 posts - 7 participants