# Importing mbox files fails at creating topics with Chinese characters due to invalid byte sequence

**URL:** <https://meta.discourse.org/t/importing-mbox-files-fails-at-creating-topics-with-chinese-characters-due-to-invalid-byte-sequence/62889>\
**Category:** Bug\
**Created:** [17.Май.2017 16:27:34 UTC](https://meta.discourse.org/t/importing-mbox-files-fails-at-creating-topics-with-chinese-characters-due-to-invalid-byte-sequence/62889 "2017-05-17T16:27:34Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![elijah](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/elijah/32/104055_2.png) [@elijah](https://meta.discourse.org/u/elijah)\
**Post date:** [26.Май.2017 00:24:37 UTC](https://meta.discourse.org/t/importing-mbox-files-fails-at-creating-topics-with-chinese-characters-due-to-invalid-byte-sequence/62889/7 "2017-05-26T00:24:37Z")

</div>

> [@yanokwa](#):
>
> I’m reporting this as a bug, because I think it’d be better if these unparseable messages were ignored by mbox.rb (maybe by looking for `?GB2312?` or `?utf-8?`

The example message you provided is not in UTF-8, hence the error seen. (UTF-8 has certain constraints that allow programs to do limited checks for conformance, “valid byte sequences”.)

The message has a content-type header, but it does not specify `charset` on that header, that normally would make figuring out the character encoding difficult, but there’s a strong hint in the subject. Following old conventions, the headers are 7-bit safe here, and the `GB2312` tells us the charset, and the following `B` tells us the transfer encoding. (The format is [MIME encoded word](https://tools.ietf.org/html/rfc2047).)

```plaintext
$ echo sKLA77DNsM274dSx0MXPorLJvK/I7bz+ | mmencode -u | iconv -f GB2312 -t utf-8
阿里巴巴会员信息采集软件

```

With charset hint, I can see the first line of the message goes something like this:

> 一锟斤拷锟斤拷锟斤拷锟斤拷桶突锟皆憋拷

---

_[View the full topic](https://meta.discourse.org/t/importing-mbox-files-fails-at-creating-topics-with-chinese-characters-due-to-invalid-byte-sequence/62889)._
