Paul_King reports character encoding issues after importing a MyBB forum into Discourse, with garbled text like ’ and  appearing, likely due to Latin1 vs UTF-8 mismatches. pfaffman confirms this is common and suggests post-processing fixes using Ruby code to force encoding conversions, cautioning to test on a staging site first. Paul_King asks if the issue can be resolved at the source by re-exporting from MyBB, and pfaffman agrees that starting over is best if the site isn’t live. Falco recommends exporting data in UTF-8 to prevent encoding problems. Paul_King later discovers that his MyBB database had mixed encodings, especially in plugin-related tables, and resolves this by updating the $config['database']['encoding'] to 'utf8mb4' in config.php and running a conversion tool in MyBB’s admin panel. He plans to re-import to Discourse to test if this fixes the character issues. He also raises a separate issue about unparsed MyCode in imported posts; riking advises starting a new thread for that topic, which Paul_King does, linking to the new threadhere.
Thanks Jay.
Is there any clever way of dealing with the issue at source - i.e re-exporting database from old forum, then re importing free of characters and mycode issues?
I went back to original MyBB installation, and found in admin control panel/Tools and Maintenance/System Health a warning
It is recommend not to use different encodings in your database. This may cause unexpected behavior or MySQL errors.
The tables are listed, and I could see most but not all were in UTF-8 format. Looked like some, particularly associated with plugins, were in an older format
Clicking a ‘Convert all’ link brought up response that /inc/config.php needed editing to support full 4 byte UTF-8
$config[‘database’][‘encoding’] = ‘utf8mb4’;
After editing config.php and trying the conversion again, all now show as matching. Will try re importing to Discourse and report back if this helps with character issues.
Not sure still how to deal with MyCode parsing though?
You didn’t include any examples or details of this - at this point, may be best to start a new thread and keep this one focused on the followup for the character encoding.