# Garbled text in CSVs with special characters downloaded from Data Resource Manager

**URL:** https://meta.discourse.org/t/csv/411280
**Category:** Bug
**Tags:** data-explorer, fixed
**Created:** [August 31, 2026, 10:10am UTC](https://meta.discourse.org/t/csv/411280 "2026-08-31T10:10:47Z")
**Posts on this page:** 1
**Showing post:** 3

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [September 9, 2026, 8:58am UTC](https://meta.discourse.org/t/csv/411280/3 "2026-09-09T08:58:45Z")

</div>

thanks for the report @singi2016cn 👍 will be fixed by

> <https://github.com/discourse/discourse/pull/43424>
>
> Reported at https://meta.discourse.org/t/csv/411280 — a Data Explorer CSV contai…ning CJK text opens as mojibake in Excel.
> 
> The bytes were always correct UTF-8. \`Content-Disposition: attachment\` writes them to disk verbatim, so the \`charset=utf-8\` header never reaches the spreadsheet and Excel falls back to the system codepage. Confirmed numerically: the exported bytes decode under CP936 to exactly the glyphs in the report.
> 
> Every CSV a person opens now starts with a BOM, and every CSV we read back tolerates one.
> 
> The same root cause was reported in 2023 for the admin user list export (https://meta.discourse.org/t/exported-csv-files-have-garbled-characters/277293) and is fixed here too.
> 
> \### Also in this PR
> 
> \- Two live bugs, independent of what we emit: an Excel-saved badge award list silently dropped its first username, and an Excel-saved emoji manifest failed every row with \`missing\_name\`.
> \- \`poll.gjs\` re-adds the BOM client-side, because the XHR text decode strips the one the server sent.
> \- \`rchardet\` is removed — nothing called \`Encodings.to\_utf8\` any more. Drops an LGPL dependency from a GPLv2 project.
> 
> \### Notes for reviewers
> 
> \- \`CSV.open(path, "w:bom|utf-8")\` does \*\*not\*\* write a BOM; Ruby honours \`bom|\` on read only. The explicit \`csv.to\_io.write\` is required.
> \- \`CSV.generate(+Encodings::BOM)\` needs the unary \`+\` — the constant is frozen and \`CSV.generate\` mutates its seed.
> \- This changes the response body of \`run.csv\`, which is reachable with an API key. Scripts parsing it should decode as \`utf-8-sig\` (Python) or equivalent. pandas, polars and duckdb need no change.
> \- It does \*\*not\*\* change the field separator. Excel on Windows takes that from the regional list-separator setting, so \`;\`-locale users still need the import wizard.
> 
> \### Plugin authors
> 
> \`paviliondev/discourse-news\` calls \`Encodings.to\_utf8\` and \`try\_utf8\` in \`lib/news/rss.rb\` and will need to inline them. It is the only consumer across all-the-plugins, all-the-themes and both custom meta repos.

---

_[View the full topic](https://meta.discourse.org/t/csv/411280)._