# Error importing from vanilla: invalid byte sequence in UTF-8

**URL:** https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320
**Category:** Migration
**Created:** [December 14, 2018, 11:30pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320 "2018-12-14T23:30:53Z")
**Posts on this page:** 6
**Page:** 2

<div class="post-metadata">

### Author: ![southpaw](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/southpaw/32/79352_2.png) [@southpaw](https://meta.discourse.org/u/southpaw)
#### Post date: [October 17, 2023, 6:55pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320/22 "2023-10-17T18:55:21Z")

</div>

Unless the file is huge, it’s pretty simply to convert to UTF-8 using the Windows Notepad, if you still have it.

I believe Vanilla Porter gives you a file in .txt.gz format. You’ll need to extract the file from that compressed format. Once you have it saved as a .txt file, you can convert it.

1. Open the .txt file in Notepad.
2. Go to File \> Save As and select UTF-8 as the encoding option.
3. Click Save.

If you don’t still have Notepad, or if the file is too large for Notepad, as @pfaffman mentioned, you can Google “convert txt to UTF-8” to find some instructions that will work with the machines and software you have available.

---

<div class="post-metadata">

### Author: ![ddeveloper](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ddeveloper/32/333791_2.png) [@ddeveloper](https://meta.discourse.org/u/ddeveloper)
#### Post date: [October 17, 2023, 8:15pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320/23 "2023-10-17T20:15:03Z")

</div>

Okay, This worked. But now I’m getting this error:

```plaintext
script/import_scripts/vanilla.rb:98:in `import_users': undefined method `[]' for nil:NilClass (NoMethodError)

admin_role_id = @roles.select { |r| r[:name] == "Administrator" }.first[:role_id]
                                                                       ^^^^^^^^^^

```

First user is Admin in database

---

<div class="post-metadata">

### Author: ![southpaw](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/southpaw/32/79352_2.png) [@southpaw](https://meta.discourse.org/u/southpaw)
#### Post date: [October 17, 2023, 8:27pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320/24 "2023-10-17T20:27:52Z")

</div>

This is exactly the error that I was getting _before_ I noticed I needed to choose `Vanilla 2` from the Source Forum Type drop-down. When leaving it with `Vanilla 1` selected, my export file had empty data tables. If you selected `Vanilla 2`, I’m afraid I don’t have much else I can offer from my limited experience with a Vanilla migration. Maybe someone else has some ideas.

What version of Vanilla is your current forum running?

---

<div class="post-metadata">

### Author: ![ddeveloper](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ddeveloper/32/333791_2.png) [@ddeveloper](https://meta.discourse.org/u/ddeveloper)
#### Post date: [October 18, 2023, 7:34pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320/26 "2023-10-18T19:34:53Z")

</div>

Vanilla Higher Logic is the current version. I upgraded to this version lately. I’ve backup of my v2.6 vanilla forum. I tried importing from old database but still same issue.

Can you confirm the UserRole entry for Admin ?

---

<div class="post-metadata">

### Author: ![southpaw](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/southpaw/32/79352_2.png) [@southpaw](https://meta.discourse.org/u/southpaw)
#### Post date: [October 18, 2023, 8:00pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320/27 "2023-10-18T20:00:09Z")

</div>

> [@ddeveloper](#):
>
> Can you confirm the UserRole entry for Admin ?

@ddeveloper, I think the answer is `16`, but I’m not certain what you’re asking!  
I’ve added some sample data from my export file below, in case it’s helpful for you to see the table structures.

> **Export data**
>
> This is what the Export file lists for the `UserRole` table, just for the `System` user and me (user 2):
> 
> ```plaintext
> Table: UserRole
> UserID:int,RoleID:int
> "1","16"
> "2","2"
> "2","8"
> "2","16"
> "2","32"
> "2","33"
> 
> ```
> 
> This is what my Export file lists for the `Role` table:  
> Table: Role
> 
> ```plaintext
> RoleID:int,Name:varchar(100),Description:varchar(500),Sort:int,Deletable:tinyint,CanSession:tinyint,PersonalInfo:tinyint
> "2","Guest","Guests can only view content. Anyone browsing the site who is not signed in is considered to be a \"Guest\".","2","0","0","0"
> "3","Unconfirmed","Users must confirm their emails before becoming full members. They get assigned to this role.","3","0","1","0"
> "4","Applicant","Users who have applied for membership\, but have not yet been accepted. They have the same permissions as guests.","4","0","1","0"
> "8","Member","Members can participate in discussions.","5","1","1","0"
> "16","Administrator","Administrators have permission to do anything.","7","1","1","0"
> "32","Moderator","Moderators have permission to edit most content.","6","1","1","0"
> "33","Ambassador","Former Ambassador in the original Community","1","1","1","0"
> 
> ```
> 
> This is what the it lists for the `User` table - just the `System` entry and me. (Some info has been redacted, but should still give you an idea of the table structure.)
> 
> ```plaintext
> Table: User
> UserID:int,Name:varchar(50),Password:varbinary(100),HashMethod:varchar(10),Photo:varchar(255),Title:varchar(100),Location:varchar(100),About:text,Email:varchar(200),ShowEmail:tinyint,CountVisits:int,CountInvitations:int,CountNotifications:int,InviteUserID:int,DiscoveryText:text,Preferences:text,Permissions:text,Attributes:text,DateSetInvitations:datetime,DateOfBirth:datetime,DateFirstVisit:datetime,DateLastActive:datetime,LastIPAddress:varchar(15),DateInserted:datetime,InsertIPAddress:varchar(15),DateUpdated:datetime,UpdateIPAddress:varchar(15),HourOffset:int,Score:float,Admin:tinyint,Confirmed:tinyint,Verified:tinyint,Banned:tinyint,Deleted:tinyint,Points:int,CountUnreadConversations:int,CountDiscussions:int,CountUnreadDiscussions:int,CountComments:int,CountDrafts:int,CountBookmarks:int
> "1","System","XXX","Random","https://XXX.com/applications/dashboard/design/images/usericon.png",\N,\N,\N,"system@stub.vanillacommunity.example","0","0","0","4",\N,\N,\N,"",\N,\N,\N,\N,\N,\N,"2022-11-18 03:39:35",\N,\N,\N,"0",\N,"2","1","0","0","0","0",\N,"1",\N,\N,\N,\N
> "2","southpaw","XXX","Vanilla","userpics/QOZBY6BY3J3T/O8C563H2DWZV.jpg",\N,\N,\N,"XXX@gmail.com","0","258","0","24",\N,\N,"{\"DashboardNav.DashboardLandingPage\":\"Settings\"\,\"DashboardNav.SectionLandingPages\":{\"Settings\":\"/dashboard/settings/layout\"\,\"Moderation\":\"/dashboard/user\"}\,\"DashboardNav.Collapsed\":[]\,\"Popup.WallComment\":\"1\"\,\"Popup.ActivityComment\":\"1\"\,\"Email.AnswerAccepted\":\"1\"\,\"Popup.AnswerAccepted\":\"1\"\,\"Email.QuestionAnswered\":\"1\"\,\"Popup.QuestionAnswered\":\"1\"\,\"Email.ConversationMessage\":\"1\"\,\"Popup.ConversationMessage\":\"1\"\,\"Email.BookmarkComment\":\"1\"\,\"Email.Mention\":\"1\"\,\"Email.ParticipateComment\":\"1\"\,\"Popup.DiscussionComment\":\"1\"\,\"Popup.BookmarkComment\":\"1\"\,\"Popup.Mention\":\"1\"\,\"Popup.ParticipateComment\":\"1\"}","{\"0\":\"Garden.Email.View\"\,\"1\":\"Garden.Settings.Manage\"\,\"2\":\"Garden.Settings.View\"\,\"3\":\"Garden.SignIn.Allow\"\,\"4\":\"Garden.Users.Add\"\,\"5\":\"Garden.Users.Edit\"\,\"6\":\"Garden.Users.Delete\"\,\"7\":\"Garden.Users.Approve\"\,\"8\":\"Garden.Activity.Delete\"\,\"9\":\"Garden.Activity.View\"\,\"10\":\"Garden.Profiles.View\"\,\"11\":\"Garden.Profiles.Edit\"\,\"12\":\"Garden.Curation.Manage\"\,\"13\":\"Garden.Moderation.Manage\"\,\"14\":\"Garden.PersonalInfo.View\"\,\"15\":\"Garden.AdvancedNotifications.Allow\"\,\"16\":\"Garden.Community.Manage\"\,\"17\":\"Garden.Uploads.Add\"\,\"18\":\"Vanilla.Tagging.Add\"\,\"19\":\"Conversations.Conversations.Add\"\,\"20\":\"Plugins.Attachments.Upload.Allow\"\,\"21\":\"Reactions.Positive.Add\"\,\"22\":\"Reactions.Negative.Add\"\,\"23\":\"Reactions.Flag.Add\"\,\"24\":\"Vanilla.Discussions.View\"\,\"25\":\"Vanilla.Discussions.Add\"\,\"26\":\"Vanilla.Discussions.Edit\"\,\"27\":\"Vanilla.Discussions.Announce\"\,\"28\":\"Vanilla.Discussions.Sink\"\,\"29\":\"Vanilla.Discussions.Close\"\,\"30\":\"Vanilla.Discussions.Delete\"\,\"31\":\"Vanilla.Comments.Add\"\,\"32\":\"Vanilla.Comments.Edit\"\,\"33\":\"Vanilla.Comments.Delete\"\,\"Vanilla.Discussions.View\":[1\,2\,3\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Add\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Edit\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Announce\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Sink\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Close\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Delete\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Comments.Add\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Comments.Edit\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Comments.Delete\":[1\,2\,4\,5\,6\,7\,8\,9]}","{\"TimeZone\":\"America/New_York\"\,\"LastLoginAttempt\":1687373198\,\"LoginRate\":1\,\"CheckedComments\":[]\,\"floodcontrol.profilecontroller.2.currentpostcount\":1\,\"\":\"2022-12-04 02:28:17\"\,\"CheckedDiscussions\":[]}",\N,"1975-09-16 00:00:00","2022-11-18 03:39:35","2023-07-17 01:06:51","A¬æ¬í¬≤","2022-11-18 03:39:35","A¬æ¬í¬≤","2023-07-11 03:36:17","A¬æ¬í¬≤","-4",\N,"1","1","1","0","0","182","3","14",\N,"83","0",\N
> 
> ```

---

<div class="post-metadata">

### Author: ![ddeveloper](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ddeveloper/32/333791_2.png) [@ddeveloper](https://meta.discourse.org/u/ddeveloper)
#### Post date: [October 18, 2023, 11:21pm UTC](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320/28 "2023-10-18T23:21:07Z")

</div>

The only difference I found was “Capitalization”. vanilla.rb script doesn’t accept lower-case letters in Role names.  
For some reason, my export file had lower-case letters in role names.  
Now this error is fixed but there are many other errors too that I’ve to deal with before I can completely migrate 😃

Now I’ve to fixed the date formating:

```plaintext
script/import_scripts/vanilla.rb:281:in `strptime': invalid date (Date::Error)

    DateTime.strptime(date, "%Y-%m-%d %H:%M:%S")

```

[Previous page](https://meta.discourse.org/t/error-importing-from-vanilla-invalid-byte-sequence-in-utf-8/104320.md?page=1)
