# Kodierung von RSS- und Atom-Feeds aus der XML-Deklaration ermitteln

**URL:** <https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242>\
**Category:** Feature\
**Tags:** pr-welcome\
**Created:** [20. August 2018 um 23:16 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242 "2018-08-20T23:16:33Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![CamilleRoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/camilleroux/32/119575_2.png) [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)\
**Post date:** [20. August 2018 um 23:16 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/1 "2018-08-20T23:16:34Z")

</div>

Hi,

When I use the RSS/ATOM import with the following RSS ([https://www.noschangements.fr/posts/index.xml](https://www.noschangements.fr/posts/index.xml)):

```xml
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0"
	xmlns:atom="http://www.w3.org/2005/Atom">
	<channel>
		<title>Posts on NosChangements</title>
		<link>https://www.noschangements.fr/posts/</link>
		<description>Recent content in Posts on NosChangements</description>
		<generator>Hugo -- gohugo.io</generator>
		<language>fr_FR</language>
		<copyright>Créé par [Camille Roux](https://www.camilleroux.com/) &lt;/br&gt; Sauf mention contraire, le contenu de ce site est sous licence [Creative Commons BY-NC-SA](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.fr)</copyright>
		<lastBuildDate>Sat, 19 Aug 2017 00:00:00 +0000</lastBuildDate>
		<atom:link href="https://www.noschangements.fr/posts/index.xml" rel="self" type="application/rss+xml"/>
		<item>
			<title>#1 Je demande à ce qu&#39;on ne prescrive plus d&#39;homéopathie</title>
			<link>https://www.noschangements.fr/posts/1-homeophatie/</link>
			<pubDate>Sat, 19 Aug 2017 00:00:00 +0000</pubDate>
			<guid>https://www.noschangements.fr/posts/1-homeophatie/</guid>
			<description>🔍 Contexte Après avoir lu plusieurs articles scientifiques dans le passé sur le sujet, je savais que l&amp;rsquo;homéopathie n&amp;rsquo;avait pas plus d&amp;rsquo;effet que l&amp;rsquo;effet placebo.
Un jour, je suis allé chez ma médecin. Etant stressé par ce que j&amp;rsquo;avais, elle m&amp;rsquo;a prescrit un médicament contre l&amp;rsquo;anxiété. Une fois à la pharmacie, par curiosité, j&amp;rsquo;ai demandé si c&amp;rsquo;était de l&amp;rsquo;homéopathie. Réponse : &amp;ldquo;oui, monsieur&amp;rdquo;
Ma docteure m&amp;rsquo;avait prescrit de l&amp;rsquo;homéopathie sans demander mon consentement ni me prévenir !</description>
		</item>
	</channel>
</rss>

```

I get the following import (The emoji and accentuated characters are bad encoded):

 ![43](https://global.discourse-cdn.com/meta/original/3X/e/5/e5e4ee78200651bb1bcfb6118a9926b99e675517.png)

---

<div class="post-metadata">

**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\
**Post date:** [21. August 2018 um 00:01 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/2 "2018-08-21T00:01:00Z")

</div>

@gerhard can you have a quick look

@CamilleRoux can you confirm you are on latest

---

<div class="post-metadata">

**Author:** ![CamilleRoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/camilleroux/32/119575_2.png) [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)\
**Post date:** [21. August 2018 um 00:20 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/3 "2018-08-21T00:20:17Z")

</div>

yep!

 ![58](https://global.discourse-cdn.com/meta/original/3X/1/2/12be2df2724b4be758faad694a0250c70d3cfdfa.png)

---

<div class="post-metadata">

**Author:** ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)\
**Post date:** [21. August 2018 um 09:10 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/4 "2018-08-21T09:10:42Z")

</div>

The server doesn’t return an encoding for the feed and unfortunately the charset detection chooses the wrong encoding.

Could you configure the webserver which serves [https://www.noschangements.fr/posts/index.xml](https://www.noschangements.fr/posts/index.xml) to set a `Content-Type` of `application/xml;charset=utf-8` for that file?

@sam I wonder if we should add a setting that lets users force an encoding for their feed. Detecting an encoding works only in same cases and short XMLs produce a lot of false positives.

Obviously the best way to solve the problem is to always serve RSS feeds with a `Content-Type` HTTP header that contains the correct `charset`.

---

<div class="post-metadata">

**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\
**Post date:** [21. August 2018 um 09:16 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/5 "2018-08-21T09:16:43Z")

</div>

I think it is legit to require whoever you are pulling from specifies the right charset, also possibly worth reporting this issue to the gem we are using.

---

<div class="post-metadata">

**Author:** ![CamilleRoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/camilleroux/32/119575_2.png) [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)\
**Post date:** [21. August 2018 um 14:20 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/6 "2018-08-21T14:20:05Z")

</div>

The charset is already specified here:

> [@CamilleRoux](#):
>
> \<?xml version=“1.0” encoding=“utf-8” standalone=“yes”?\>

By the way I reported the idea of adding the header to Hugo team.

---

<div class="post-metadata">

**Author:** ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)\
**Post date:** [21. August 2018 um 15:19 UTC](https://meta.discourse.org/t/detect-encoding-of-rss-and-atom-feeds-from-the-xml-declaration/95242/7 "2018-08-21T15:19:41Z")

</div>

Yeah, we currently aren’t relying on the the XML encoding declaration. I guess the system could try to detect the correct encoding as described in [XML - Autodetection of Character Encodings](https://www.w3.org/TR/REC-xml/#sec-guessing) before it finally uses the rchardet gem to detect the most likeliest encoding.

I’m adding the #pr-welcome tag in case someone from the community wants to improve the behaviour.