Migrating vBulletin 5 database - Import script errors

Final update I guess.

We finally managed to align everything (infra, code, users, mods, etc) and migrate. It happened yesterday. I won’t link the community as I can’t recall if it’s allowed and anyway it’s quite a well known community in Italy.

These are the numbers we do on average in 30 days, bots already filtered.

There was of course quite the pressure onto the team of volunteers that took care of it, and now it’s not finished yet as we are still ironing out the custom theme and some of discourse background settings (I’ll have to open lots of topics asking for help/clarification/directions it seems).

Our script managed to migrate everything we wanted:

  • users
  • user groups
  • moderator/ban/admin status
  • private messages
  • categories
  • topic
  • replies

and so on and so forth. We also integrate the cooking in the migration process itself as we have some customization in vbulletin to allow for embedding tweets, youtube videos and other stuff that wouldn’t translate well with how discourse cook them by default.

We run the tests on a 4vcore/8GB and the whole migration completed in around 7-8 hours
For production we raised enough our patreon to afford an 8 vcore / 30 GB and the whole thing took 4 hours.

We did a live of the migration, with a couple of false start included (of course :stuck_out_tongue:) and some music in background. We have had quite some fun.

You can see the details of the number of topics/post and the timing in the screenshot.
The three timing are: time to read, time to cook, time to write.

It has been an exhausting but exilarating adventure and, @pfaffman, believe me, you dodged a bullet when I decided to not employ your help.

As of today, the count of the hours only of MY time for this project roughly estimated for £ 25k :rofl:
I’m not counting the time the other three people dedicated to it in the past 2 months or so, often working on it late at night.

We are still running some post-migration script, one that import all avatars and another that create all the permalink redirect so that link written inside replies that still point at the old URL format will actually redirect properly. I expect those to finish in the next 24 hours.

We’ll discuss, in a couple of weeks or so, if we can clean up the repository of our script and offer it open source. I can’t take that decision alone of course.

Edit: Just adding the complete migration of all user avatars + permalinks of internal references to topics/categories

Run after the main data migration.

Cheers

3 Likes