Error importing from vanilla: invalid byte sequence in UTF-8

Migration
22

Unless the file is huge, it’s pretty simply to convert to UTF-8 using the Windows Notepad, if you still have it.

I believe Vanilla Porter gives you a file in .txt.gz format. You’ll need to extract the file from that compressed format. Once you have it saved as a .txt file, you can convert it.

  1. Open the .txt file in Notepad.
  2. Go to File > Save As and select UTF-8 as the encoding option.
  3. Click Save.

If you don’t still have Notepad, or if the file is too large for Notepad, as @pfaffman mentioned, you can Google “convert txt to UTF-8” to find some instructions that will work with the machines and software you have available.

4 Likes
23

Okay, This worked. But now I’m getting this error:

script/import_scripts/vanilla.rb:98:in `import_users': undefined method `[]' for nil:NilClass (NoMethodError)

admin_role_id = @roles.select { |r| r[:name] == "Administrator" }.first[:role_id]
                                                                       ^^^^^^^^^^

First user is Admin in database

24

This is exactly the error that I was getting before I noticed I needed to choose Vanilla 2 from the Source Forum Type drop-down. When leaving it with Vanilla 1 selected, my export file had empty data tables. If you selected Vanilla 2, I’m afraid I don’t have much else I can offer from my limited experience with a Vanilla migration. Maybe someone else has some ideas.

What version of Vanilla is your current forum running?

5 Likes
26

Vanilla Higher Logic is the current version. I upgraded to this version lately. I’ve backup of my v2.6 vanilla forum. I tried importing from old database but still same issue.

Can you confirm the UserRole entry for Admin ?

27

@ddeveloper, I think the answer is 16, but I’m not certain what you’re asking!
I’ve added some sample data from my export file below, in case it’s helpful for you to see the table structures.

Export data

This is what the Export file lists for the UserRole table, just for the System user and me (user 2):

Table: UserRole
UserID:int,RoleID:int
"1","16"
"2","2"
"2","8"
"2","16"
"2","32"
"2","33"

This is what my Export file lists for the Role table:
Table: Role

RoleID:int,Name:varchar(100),Description:varchar(500),Sort:int,Deletable:tinyint,CanSession:tinyint,PersonalInfo:tinyint
"2","Guest","Guests can only view content. Anyone browsing the site who is not signed in is considered to be a \"Guest\".","2","0","0","0"
"3","Unconfirmed","Users must confirm their emails before becoming full members. They get assigned to this role.","3","0","1","0"
"4","Applicant","Users who have applied for membership\, but have not yet been accepted. They have the same permissions as guests.","4","0","1","0"
"8","Member","Members can participate in discussions.","5","1","1","0"
"16","Administrator","Administrators have permission to do anything.","7","1","1","0"
"32","Moderator","Moderators have permission to edit most content.","6","1","1","0"
"33","Ambassador","Former Ambassador in the original Community","1","1","1","0"

This is what the it lists for the User table - just the System entry and me. (Some info has been redacted, but should still give you an idea of the table structure.)

Table: User
UserID:int,Name:varchar(50),Password:varbinary(100),HashMethod:varchar(10),Photo:varchar(255),Title:varchar(100),Location:varchar(100),About:text,Email:varchar(200),ShowEmail:tinyint,CountVisits:int,CountInvitations:int,CountNotifications:int,InviteUserID:int,DiscoveryText:text,Preferences:text,Permissions:text,Attributes:text,DateSetInvitations:datetime,DateOfBirth:datetime,DateFirstVisit:datetime,DateLastActive:datetime,LastIPAddress:varchar(15),DateInserted:datetime,InsertIPAddress:varchar(15),DateUpdated:datetime,UpdateIPAddress:varchar(15),HourOffset:int,Score:float,Admin:tinyint,Confirmed:tinyint,Verified:tinyint,Banned:tinyint,Deleted:tinyint,Points:int,CountUnreadConversations:int,CountDiscussions:int,CountUnreadDiscussions:int,CountComments:int,CountDrafts:int,CountBookmarks:int
"1","System","XXX","Random","https://XXX.com/applications/dashboard/design/images/usericon.png",\N,\N,\N,"system@stub.vanillacommunity.example","0","0","0","4",\N,\N,\N,"",\N,\N,\N,\N,\N,\N,"2022-11-18 03:39:35",\N,\N,\N,"0",\N,"2","1","0","0","0","0",\N,"1",\N,\N,\N,\N
"2","southpaw","XXX","Vanilla","userpics/QOZBY6BY3J3T/O8C563H2DWZV.jpg",\N,\N,\N,"XXX@gmail.com","0","258","0","24",\N,\N,"{\"DashboardNav.DashboardLandingPage\":\"Settings\"\,\"DashboardNav.SectionLandingPages\":{\"Settings\":\"/dashboard/settings/layout\"\,\"Moderation\":\"/dashboard/user\"}\,\"DashboardNav.Collapsed\":[]\,\"Popup.WallComment\":\"1\"\,\"Popup.ActivityComment\":\"1\"\,\"Email.AnswerAccepted\":\"1\"\,\"Popup.AnswerAccepted\":\"1\"\,\"Email.QuestionAnswered\":\"1\"\,\"Popup.QuestionAnswered\":\"1\"\,\"Email.ConversationMessage\":\"1\"\,\"Popup.ConversationMessage\":\"1\"\,\"Email.BookmarkComment\":\"1\"\,\"Email.Mention\":\"1\"\,\"Email.ParticipateComment\":\"1\"\,\"Popup.DiscussionComment\":\"1\"\,\"Popup.BookmarkComment\":\"1\"\,\"Popup.Mention\":\"1\"\,\"Popup.ParticipateComment\":\"1\"}","{\"0\":\"Garden.Email.View\"\,\"1\":\"Garden.Settings.Manage\"\,\"2\":\"Garden.Settings.View\"\,\"3\":\"Garden.SignIn.Allow\"\,\"4\":\"Garden.Users.Add\"\,\"5\":\"Garden.Users.Edit\"\,\"6\":\"Garden.Users.Delete\"\,\"7\":\"Garden.Users.Approve\"\,\"8\":\"Garden.Activity.Delete\"\,\"9\":\"Garden.Activity.View\"\,\"10\":\"Garden.Profiles.View\"\,\"11\":\"Garden.Profiles.Edit\"\,\"12\":\"Garden.Curation.Manage\"\,\"13\":\"Garden.Moderation.Manage\"\,\"14\":\"Garden.PersonalInfo.View\"\,\"15\":\"Garden.AdvancedNotifications.Allow\"\,\"16\":\"Garden.Community.Manage\"\,\"17\":\"Garden.Uploads.Add\"\,\"18\":\"Vanilla.Tagging.Add\"\,\"19\":\"Conversations.Conversations.Add\"\,\"20\":\"Plugins.Attachments.Upload.Allow\"\,\"21\":\"Reactions.Positive.Add\"\,\"22\":\"Reactions.Negative.Add\"\,\"23\":\"Reactions.Flag.Add\"\,\"24\":\"Vanilla.Discussions.View\"\,\"25\":\"Vanilla.Discussions.Add\"\,\"26\":\"Vanilla.Discussions.Edit\"\,\"27\":\"Vanilla.Discussions.Announce\"\,\"28\":\"Vanilla.Discussions.Sink\"\,\"29\":\"Vanilla.Discussions.Close\"\,\"30\":\"Vanilla.Discussions.Delete\"\,\"31\":\"Vanilla.Comments.Add\"\,\"32\":\"Vanilla.Comments.Edit\"\,\"33\":\"Vanilla.Comments.Delete\"\,\"Vanilla.Discussions.View\":[1\,2\,3\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Add\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Edit\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Announce\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Sink\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Close\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Discussions.Delete\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Comments.Add\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Comments.Edit\":[1\,2\,4\,5\,6\,7\,8\,9]\,\"Vanilla.Comments.Delete\":[1\,2\,4\,5\,6\,7\,8\,9]}","{\"TimeZone\":\"America/New_York\"\,\"LastLoginAttempt\":1687373198\,\"LoginRate\":1\,\"CheckedComments\":[]\,\"floodcontrol.profilecontroller.2.currentpostcount\":1\,\"\":\"2022-12-04 02:28:17\"\,\"CheckedDiscussions\":[]}",\N,"1975-09-16 00:00:00","2022-11-18 03:39:35","2023-07-17 01:06:51","A¾’²","2022-11-18 03:39:35","A¾’²","2023-07-11 03:36:17","A¾’²","-4",\N,"1","1","1","0","0","182","3","14",\N,"83","0",\N
3 Likes
28

The only difference I found was “Capitalization”. vanilla.rb script doesn’t accept lower-case letters in Role names.
For some reason, my export file had lower-case letters in role names.
Now this error is fixed but there are many other errors too that I’ve to deal with before I can completely migrate :smiley:

Now I’ve to fixed the date formating:

script/import_scripts/vanilla.rb:281:in `strptime': invalid date (Date::Error)

    DateTime.strptime(date, "%Y-%m-%d %H:%M:%S")
2 Likes