# Better email reply parsing 📧

**URL:** https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495
**Category:** Feature
**Tags:** email, feedback
**Created:** [December 9, 2015, 6:49pm UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495 "2015-12-09T18:49:18Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [December 9, 2015, 6:49pm UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/1 "2015-12-09T18:49:18Z")

</div>

There are many open bugs on meta related to incorrect email reply parsing and I have been looking into them since past few days.

Almost everytime the fix is to correct/modify something in GitHub’s [email\_reply\_parser](https://github.com/github/email_reply_parser) library.

The GitHub’s email\_reply\_parser library is now abandoned. The last commit was over a year ago, and there are many open issues / pull requests.

I propose that we include the email\_reply\_parser in core Discourse email library so that we can customize it as per our requirements, without depending on GitHub’s library.

I have laid the groundwork for this, and in process fixed these bugs:

> [@Email parsing reply not correctly stripping \`----Original message----\` marker](https://meta.discourse.org/t/email-parsing-reply-not-correctly-stripping-original-message-marker/24112):
>
> Email parsing reply not correctly handling ----Original message---- marker. Here is a modified example (to protect privacy): I have two examples of this now with the same user. I’m willing to send via PM the raw original email replies to a one of the Discourse Team to help resolve this. Cheers, Dean.

[https://meta.discourse.org/t/html-email-signature-not-being-stripped-out-of-notification-reply/21351](https://meta.discourse.org/t/html-email-signature-not-being-stripped-out-of-notification-reply/21351)

> [@Email-created reply including reply preamble](https://meta.discourse.org/t/email-created-reply-including-reply-preamble/25472):
>
> Reproduce: Reply using Gmail to a post notification. bulbGet a separate rejection error because your reply is less than the default 20 character minimum. facepunchReply to your first attempt to reply (which still is addressed to Discourse) with a longer response. angryExpected: Only the response content appears. Actual: Your email address is exposed. astonished Pastebin of original message content available to those who request.

> [@Malformed bullet causes emailed in topic to be incorrectly truncated](https://meta.discourse.org/t/malformed-bullet-causes-emailed-in-topic-to-be-incorrectly-truncated/25374):
>
> I know that a line with two dashes truncates everything below it. -- But it turns out that a line with a single dash immediately followed by text (no space) will also truncate everything below it. See screenshots.

Here is the PR for the same:

[https://github.com/discourse/discourse/pull/3918](https://github.com/discourse/discourse/pull/3918)

Looking forward to hear [@team](https://meta.discourse.org/groups/team)’s feedback/suggestions.

---

<div class="post-metadata">

### Author: ![eviltrout](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/eviltrout/32/5275_2.png) [@eviltrout](https://meta.discourse.org/u/eviltrout)
#### Post date: [December 9, 2015, 8:41pm UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/2 "2015-12-09T20:41:14Z")

</div>

Wow I did not realize that library was only about 280 lines of code!

One downside with importing it this way is you are not importing their tests. If we regress on something they already have a test for, we are not going to know are we?

Maybe it makes more sense to fork it as discourse-email-parser, and add your new functionality and tests to the suite in that project?

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [December 9, 2015, 9:08pm UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/3 "2015-12-09T21:08:03Z")

</div>

I agree 💯%. Forking is a better idea.

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [December 10, 2015, 7:45pm UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/4 "2015-12-10T19:45:54Z")

</div>

Okay, I just published a new gem **discourse\_email\_parser** 📨

Here is the GitHub repo: [GitHub - discourse/discourse\_email\_parser: Small library to parse plain text email content · GitHub](https://github.com/discourse/discourse_email_parser)

and [RubyGems.org](http://RubyGems.org) page: [discourse\_email\_parser | RubyGems.org | your community gem host](https://rubygems.org/gems/discourse_email_parser)

Updated the [PR](https://github.com/discourse/discourse/pull/3918) to use `discourse_email_parser` instead of `email_reply_parser` gem.

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [December 11, 2015, 2:31pm UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/5 "2015-12-11T14:31:12Z")

</div>

This change is now live, @zogstrip just merged the PR! 🎉

---

<div class="post-metadata">

### Author: ![fefrei](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fefrei/32/119538_2.png) [@fefrei](https://meta.discourse.org/u/fefrei)
#### Post date: [February 4, 2016, 9:41am UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/6 "2016-02-04T09:41:56Z")

</div>

It looks like the parser is not yet trying to detect the “On `<date>`, `<name>` wrote”-line in any non-english language:

> <https://github.com/discourse/discourse_email_parser/blob/dfc6031cfce718e4d0cadd9a52a72be0016e2c55/lib/discourse_email_parser.rb#L89>

Is this correct? How can we contribute translations for this to at least cover the most common languages?

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [February 4, 2016, 9:52am UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/7 "2016-02-04T09:52:06Z")

</div>

`discourse_email_parser` is now replaced by `email_reply_trimmer`

[https://github.com/discourse/email\_reply\_trimmer](https://github.com/discourse/email_reply_trimmer)

and it covers common languages 😉

> <https://github.com/discourse/email_reply_trimmer/blob/c7f36f34afdf3b40b47ec89f01dc4a1a7b8eb194/lib/email_reply_trimmer/embedded_email_matcher.rb#L13-L26>

---

<div class="post-metadata">

### Author: ![fefrei](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fefrei/32/119538_2.png) [@fefrei](https://meta.discourse.org/u/fefrei)
#### Post date: [February 4, 2016, 9:58am UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/8 "2016-02-04T09:58:59Z")

</div>

That list might soon include German:

[https://github.com/discourse/email\_reply\_trimmer/pull/1](https://github.com/discourse/email_reply_trimmer/pull/1)

---

<div class="post-metadata">

### Author: ![techAPJ](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/techapj/32/342990_2.png) [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [February 5, 2016, 10:06am UTC](https://meta.discourse.org/t/better-email-reply-parsing-e-mail/36495/9 "2016-02-05T10:06:44Z")

</div>

This topic was automatically closed after 24 hours. New replies are no longer allowed.
