# Parsing RSS feed missing quotes + apostrophes

**URL:** https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880
**Category:** Bug
**Created:** [5월 19, 2018, 12:58오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880 "2018-05-19T12:58:35Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![Patrick\_David](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/patrick_david/32/119657_2.png) [@Patrick\_David](https://meta.discourse.org/u/Patrick_David)
#### Post date: [5월 19, 2018, 12:58오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/1 "2018-05-19T12:58:35Z")

</div>

Hi all!

We’ve recently upgraded to `2.0.0.beta10~git49.9f422c93f6` and our latest RSS pull for comment embedding nuked apostrophes and quotes from the posts:

 ![](https://global.discourse-cdn.com/meta/original/3X/c/7/c740aec67413a8d690c15917ac587dfd1daa9b94.png)

The image example is from this page:

> **[An About Page and Help](https://discuss.pixls.us/t/an-about-page-and-help/102)**
>
> !\[\](upload://49NA5Lf5qZNUlieQIx01uo2csfG.jpeg) An About Page and Help A little more about site I’ve started working a bit on the “About” page for the site. I wanted a place to highlight the...

The feed it’s pulling from is here: [PIXLS.US](https://pixls.us/feed.xml)

Search on “an about page and help” to get to the relevant section.

y u hate typographical marks?! 😃

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [5월 19, 2018, 1:05오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/2 "2018-05-19T13:05:02Z")

</div>

@techAPJ didn’t you make changes to the feed poller recently?

---

<div class="post-metadata">

### Author: ![darix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/darix/32/114280_2.png) [@darix](https://meta.discourse.org/u/darix)
#### Post date: [5월 20, 2018, 10:49오전 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/3 "2018-05-20T10:49:16Z")

</div>

So this seems to be 2 separate issues:

1. when upgrading from 2.0.0.beta9+git0 to the version mentioned above discourse decided it needs to refetch/rerender a lot of older posts from the RSS feed. This is how we noticed the 2nd bug
2. It seems we lost at least all typography markers. also in our recent posts. The import category can be seen here [PIXLS.US - discuss.pixls.us](https://discuss.pixls.us/c/pixls-us)

It smells a bit like “RSS feed despite being sent with the correct headers is not seen/read as utf-8 encoded string so that the reencoding with the replace option strips utf-8 encoded chars.”

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [5월 20, 2018, 12:23오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/4 "2018-05-20T12:23:48Z")

</div>

Thanks for bringing this to our notice. I have [reverted](https://github.com/discourse/discourse/commit/1841dd48dc5c15025dac8b6054c35045334a2a55) the UTF-8 encode related changes I pushed few days ago.

Updating to latest version will normalize the behaviour.

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [5월 20, 2018, 1:36오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/7 "2018-05-20T13:36:31Z")

</div>

> [@darix](#):
>
> when upgrading from 2.0.0.beta9+git0 to the version mentioned above discourse decided it needs to refetch/rerender a lot of older posts from the RSS feed.

Since the content was updated/changed the topic got updated as per [this code](https://github.com/discourse/discourse/blob/0cc4b4218038fd5fb8a7007de4329f46ed1bbc90/app/models/topic_embed.rb#L74-L84). Now that I reverted the code, the topic will be updated again with proper formatting.

---

<div class="post-metadata">

### Author: ![darix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/darix/32/114280_2.png) [@darix](https://meta.discourse.org/u/darix)
#### Post date: [5월 20, 2018, 2:13오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/8 "2018-05-20T14:13:22Z")

</div>

ok I patched out the “not recently polled” check for a moment. triggered the sidekig job and all our posts are good again.

Why was this reencode added in the beginning? I will debug later why the raw\_feed string isnt utf-8 encoded. then the reencode should have been a noop no?

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [5월 20, 2018, 2:18오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/9 "2018-05-20T14:18:58Z")

</div>

> [@darix](#):
>
> Why was this reencode added in the beginning?

Because in some cases of bad (not supported) encoding we were seeing job exceptions in error logs.

> [@darix](#):
>
> I will debug later why the raw\_feed string isnt utf-8 encoded. then the reencode should have been a noop no?

Yes, if the string is proper UTF-8 encoded already, then the `encode` logic shouldn’t have come into action.

---

<div class="post-metadata">

### Author: ![darix](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/darix/32/114280_2.png) [@darix](https://meta.discourse.org/u/darix)
#### Post date: [5월 20, 2018, 2:20오후 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/10 "2018-05-20T14:20:32Z")

</div>

might depend on the locale/lang environment of the sidekiq job. maybe i should add LC\_ALL=en\_US.UTF-8 and LANG=en\_US.UTF-8 in my service file.

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [5월 31, 2018, 5:22오전 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/11 "2018-05-31T05:22:44Z")

</div>

@gerhard sent a PR with proper fix:

[https://github.com/discourse/discourse/pull/5893](https://github.com/discourse/discourse/pull/5893)

Closing this topic for now. Please create a new topic if the problem persists.

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [5월 31, 2018, 5:22오전 UTC](https://meta.discourse.org/t/parsing-rss-feed-missing-quotes-apostrophes/87880/12 "2018-05-31T05:22:47Z")

</div>


