# Import scripts vbulletin5.rb

**URL:** https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531
**Category:** Self-hosting
**Created:** [November 4, 2019, 1:17pm UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531 "2019-11-04T13:17:03Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![INSTAR](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/instar/32/359540_2.png) [@INSTAR](https://meta.discourse.org/u/INSTAR)
#### Post date: [November 4, 2019, 1:17pm UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/1 "2019-11-04T13:17:03Z")

</div>

I am using the official [import script for vBulletin v5](https://github.com/discourse/discourse/blob/master/script/import_scripts/vbulletin5.rb) and I ran into an issue.

My vBulletin forum structure is like:

```plaintext
/forum/
/forum/sub-category1/
/forum/sub-category1/thread1/
/forum/sub-category1/thread2/
/forum/sub-category2/
/forum/sub-category2/thread1/
/forum/sub-category2/thread2/

```

When I run the import script this results in two categories in Discourse without any threads:

```plaintext
/c/sub-category1
/c/sub-category2

```

I checked out a different vBulletin forum and noticed that they do not nest their URLs. I now assume that this is the change that I have to make to my Forum before I am able to use the script:

```plaintext
/forum/
/forum/sub-category1/
/forum/sub-category2/
/forum/thread1a/
/forum//thread2a/
/forum/thread1b/
/forum//thread2b/

```

Afterwards, I would have to manually assign categories to every thread again.

Does someone have experience using this script?

Is there maybe something that jumps to your eyes inside the script, that can be changed so that it can handle a more nested structure? I might be on a wrong path completely.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [November 4, 2019, 2:46pm UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/2 "2019-11-04T14:46:52Z")

</div>

By “thread” do you mean discourse topic? Are the topics being created but not in the correct categories?

---

<div class="post-metadata">

### Author: ![INSTAR](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/instar/32/359540_2.png) [@INSTAR](https://meta.discourse.org/u/INSTAR)
#### Post date: [November 4, 2019, 4:40pm UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/3 "2019-11-04T16:40:06Z")

</div>

Hey Jay,

thank you for your reply.

Ah yes - I mean Discourse Topics. There is nothing there except those empty categories. And all the users are imported successfully.

Mike

 ![Untitled](https://global.discourse-cdn.com/meta/original/3X/f/8/f8666ac25e58bb6950f3b5eeea162cdbc769d944.png)

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [November 4, 2019, 5:00pm UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/4 "2019-11-04T17:00:27Z")

</div>

Something is keeping topics and posts from getting imported. I’d think that you’d see errors when it ran. Topics and posts get created after users. I believe that topics and posts are in separate functions. The importer should show you progress on those just like it did for the users.

---

<div class="post-metadata">

### Author: ![INSTAR](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/instar/32/359540_2.png) [@INSTAR](https://meta.discourse.org/u/INSTAR)
#### Post date: [November 5, 2019, 3:56am UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/5 "2019-11-05T03:56:03Z")

</div>

There are no Error messages during the import - only a few warnings for invalid user email addresses when the `create_users` function runs. But the user import is successful in the end. Here is the log (I removed the warnings as they contained customer email addresses):

> <https://gist.github.com/mpolinowski/ae6447b5b95a412856c51aed6556ba84>

```plaintext
importing posts...
5 / 5 (100.0%) [290527 items/min]                                                            
importing attachments...   
20182 / 5 (403640.0%)

```

Here is where I think it only sees my five categories as posts and imports them. And does not know what to do with all the attachments for the actual topics that weren’t imported?

**Disclaimer** :

Initially, the import failed because the vBulletin database I was given did not have a table `customprofilepic` - I added an empty table here.

And there were no custom avatars - so the `AVATAR_DIR`, referenced in the import script, is just an empty folder I created. I commented out the following lines in the import script that were referencing those assets:

```ruby
create_users(users, total: user_count, offset: offset) do |user|
        username = @htmlentities.decode(user["username"]).strip
        {
          id: user["userid"],
          name: username,
          username: username,
          email: user["email"].presence || fake_email,
          admin: user['admin'] == 1,
          password: user["password"],
          website: user["homepage"].strip,
          title: @htmlentities.decode(user["usertitle"]).strip,
          primary_group_id: group_id_from_imported_group_id(user["usergroupid"]),
          created_at: parse_timestamp(user["joindate"])
          # post_create_action: proc do |u|
          # @old_username_to_new_usernames[user["username"]] = u.username
          # import_profile_picture(user, u)
          # import_profile_background(user, u)
          # end
        }
      end

```

But to my understanding, those changes only affect the user creation. And the users are created successfully.

This is the full import script that I am using:

> <https://gist.github.com/mpolinowski/0ef4df888720ad4a92c4bf5edac2e076>

---

<div class="post-metadata">

### Author: ![INSTAR](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/instar/32/359540_2.png) [@INSTAR](https://meta.discourse.org/u/INSTAR)
#### Post date: [November 5, 2019, 4:10am UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/6 "2019-11-05T04:10:19Z")

</div>

The script collects all topics from the `node` table within the vBulletin database. Which is present and holding about 34k postings. The column names are identical to what the script is referencing:

> <https://gist.github.com/mpolinowski/2d72874a41eb7b8dbbd2afe2da4ad279>

It looks as if it should just work 🙄

---

<div class="post-metadata">

### Author: ![INSTAR](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/instar/32/359540_2.png) [@INSTAR](https://meta.discourse.org/u/INSTAR)
#### Post date: [November 5, 2019, 9:35am UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/7 "2019-11-05T09:35:11Z")

</div>

The import script searches for `contenttypeid=23` and `contenttypeid=22` to create postings.

By skimming through the `node` table I can say, that the **23** stands for a posting with an attachment and **22** should be the attachment itself.

The majority of the posts are of `contenttypeid=21` - postings without attachments. And I don’t see an import for those. Or is this covered by:

```ruby
post_count = mysql_query("SELECT COUNT(nodeid) cnt FROM #{DBPREFIX}node WHERE parentid NOT IN (
SELECT nodeid FROM #{DBPREFIX}node WHERE contenttypeid=23 ) AND contenttypeid=22;").first["cnt"]

```

This goes through everything that is left over after the import is done with 23/22. But there are more content type IDs beside 23/22/21 - so this would get those as well.

* * *

And categories are imported relative to a `ROOT_NODE` that was preset to **2**. I might have to set it to 3 to support my “3-level nested” URLs?

UPDATE: I just re-ran the import with ROOT\_NODE=3 and and it did not change anything

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [November 5, 2019, 9:50am UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/8 "2019-11-05T09:50:24Z")

</div>

It sounds like your vBulletin is different from what the script expects. Before you can get the posts to import, you’ll need to figure out why none of the topics got created.

---

<div class="post-metadata">

### Author: ![INSTAR](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/instar/32/359540_2.png) [@INSTAR](https://meta.discourse.org/u/INSTAR)
#### Post date: [November 5, 2019, 10:49am UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/9 "2019-11-05T10:49:50Z")

</div>

Ok. It is the `contenttypeid` that is causing the issue:

```ruby
def import_categories
    puts "", "importing top level categories..."

    categories = mysql_query("SELECT nodeid AS forumid, title, description, displayorder, parentid
	      FROM #{DBPREFIX}node
          WHERE parentid=#{ROOT_NODE}
        UNION
          SELECT nodeid, title, description, displayorder, parentid
          FROM #{DBPREFIX}node
          WHERE contenttypeid = 20
            AND parentid IN (SELECT nodeid FROM #{DBPREFIX}node WHERE parentid=#{ROOT_NODE})").to_a

```

For example the `import_categories` function uses `contenttypeid = 23` which is the same ID the script uses for postings. The ID for categories is `20`. By changing this inside the script I am finally able to see my sub-categories / topics 🙌

 ![Untitled](https://global.discourse-cdn.com/meta/original/3X/1/c/1c8334e3bd53855eca84f6e59d44c9f0556130aa.png)

Now all that is missing are the threads / posts

---

<div class="post-metadata">

### Author: ![michaeld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/michaeld/32/1594_2.png) [@michaeld](https://meta.discourse.org/u/michaeld)
#### Post date: [April 20, 2020, 4:28pm UTC](https://meta.discourse.org/t/import-scripts-vbulletin5-rb/132531/10 "2020-04-20T16:28:55Z")

</div>

> [@INSTAR](#):
>
> The import script searches for `contenttypeid=23` and `contenttypeid=22` to create postings.

I have made a pull request with some updates to the VBulletin5 importer. One of the most important changes is that it got rid of these hard coded ID’s.

There is also better support for uploads and the importer now imports tags and creates permalinks for topics and (sub)categories.
