# Email reply parsing

**URL:** https://meta.discourse.org/t/email-reply-parsing/41597
**Category:** Bug
**Tags:** email
**Created:** [March 26, 2016, 12:03pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597 "2016-03-26T12:03:58Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![steko](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@steko](https://meta.discourse.org/u/steko)
#### Post date: [March 26, 2016, 12:03pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597/1 "2016-03-26T12:03:58Z")

</div>

I’ve finally setup reply by email for my Discourse forum, but unfortunately parsing of replies seems suboptimal and the original message is not stripped. My forum is in Italian, but that should be accounted for, at least judging by some earlier bug reports about this same issue. I’ve seen a few other topics here on Meta (and there are probably older ones as well):

> [@Reply by email parsing links from previous messages](https://meta.discourse.org/t/reply-by-email-parsing-links-from-previous-messages/34634):
>
> Post a message with a link. Somebody replies by email to it. When Discourse handles the email reply, it improperly includes the link from the previous message. Here’s an example:

> [@Emails reply shows date, emails and «wrote» from gmail](https://meta.discourse.org/t/emails-reply-shows-date-emails-and-wrote-from-gmail/30747):
>
> If you reply using email, gmail add El 2/7/2015 3:11 p. m., “MiguelAngelLV” [hacklabalmeria.discourse@gmail.com](mailto:hacklabalmeria.discourse@gmail.com) escribió: The format is: datetime, nick of last user, email address that send the email, and “wrote” o “escribió” in spanish, and this message is show in the reply, [example](http://foro.hacklabalmeria.net/t/estamos-mandado-emails-por-encima-de-nuestras-posibilidades/5653/4?u=miguelangellv) I know that “this is not a bug of Discourse”, but gmail is a popular email. Would be added a regular expression or something to remove the message?

I don’t use Gmail, that probably accounts for 50% of my users alone and should be covered, but I wonder if there’s a more robust way to support quirks of other email clients (e.g. mobile email clients, since replying to email from mobile is one of the use cases that Discourse wants to support, I think). Of course I can just suggest users to strip entirely the original message or leave only short quotes if they like, but … I’m sure Discourse can handle this better!

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 26, 2016, 12:05pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597/2 "2016-03-26T12:05:53Z")

</div>

We need specific examples of failures, PM full source of emails to @zogstrip

---

<div class="post-metadata">

### Author: ![steko](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@steko](https://meta.discourse.org/u/steko)
#### Post date: [March 26, 2016, 12:23pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597/3 "2016-03-26T12:23:54Z")

</div>

Ok, will do. As a test, this is a reply from a webmail client.

> Il 26 marzo 2016 alle 13.16 Sam Saffron [info@discourse.org](mailto:info@discourse.org) ha scritto:
> 
> We need specific examples of failures, PM full source of emails to  
> @zogstrip
> 
> Posted by sam on 03/26/2016
> 
> * * *
> 
> [Visit Topic](https://meta.discourse.org/t/email-reply-parsing/41597/2)  
> or reply to this email to respond
> 
> To stop receiving notifications for this particular topic, [click  
> here](https://meta.discourse.org/t/email-reply-parsing/41597/unsubscribe).  
> To unsubscribe from these emails, change your [user  
> preferences](https://meta.discourse.org/my/preferences)

Stefano Costa @stekosteko  
Editor, Journal of Open Archaeology Data

---

<div class="post-metadata">

### Author: ![DeanMarkTaylor](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/deanmarktaylor/32/102462_2.png) [@DeanMarkTaylor](https://meta.discourse.org/u/DeanMarkTaylor)
#### Post date: [March 26, 2016, 1:29pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597/4 "2016-03-26T13:29:17Z")

</div>

This might be unrelated but…

Make sure you are on the latest version.

There was a bug apparently fixed last week where the `"<details>"` section was not being removed, that contained the previous posts.

I’m currently experiencing this one, but I haven’t updated to see if the fix does resolve the issue.

---

<div class="post-metadata">

### Author: ![steko](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@steko](https://meta.discourse.org/u/steko)
#### Post date: [March 26, 2016, 2:28pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597/5 "2016-03-26T14:28:24Z")

</div>

Yes, I saw the `<details>`-related bug but I upgrade daily to the latest version so I don’t think it affected me when testing reply via email today. Thanks for pointing this out however!

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 27, 2016, 12:52am UTC](https://meta.discourse.org/t/email-reply-parsing/41597/6 "2016-03-27T00:52:20Z")

</div>

The Italian dates seem to be tripping the email parser, we need a smarter heuristic

---

<div class="post-metadata">

### Author: ![steko](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@steko](https://meta.discourse.org/u/steko)
#### Post date: [March 29, 2016, 9:05am UTC](https://meta.discourse.org/t/email-reply-parsing/41597/7 "2016-03-29T09:05:45Z")

</div>

Today I tried removing the original message from a reply entirely, but it seems to miss also the signature at the bottom of the message, it’s kept as is (but that shouldn’t be language-dependent) 😕

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [March 29, 2016, 10:19am UTC](https://meta.discourse.org/t/email-reply-parsing/41597/8 "2016-03-29T10:19:44Z")

</div>

> [@steko](#):
>
> but it seems to miss also the signature at the bottom of the message,

We **do not** remove user signatures.

Since there is no standards around user signatures, there is no **easy** way to even get it to work for 75% of the emails.

The [best solution](https://github.com/mailgun/talon) I’ve found so far requires machine learning and **lots** of training samples. It gets much harder when you have to support more than one locale…

* * *

EDIT: I already added support for italian replies, but did push a new version of the gem. I just did, you should update 😉

[https://github.com/discourse/discourse/commit/480e8d0dc7c5627b36ad3151719c67780fa3c5b6](https://github.com/discourse/discourse/commit/480e8d0dc7c5627b36ad3151719c67780fa3c5b6)

---

<div class="post-metadata">

### Author: ![steko](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@steko](https://meta.discourse.org/u/steko)
#### Post date: [March 29, 2016, 10:56am UTC](https://meta.discourse.org/t/email-reply-parsing/41597/9 "2016-03-29T10:56:59Z")

</div>

Oh, I was under the impression that signatures starting with “–\n” or “–  
\n” were already removed by the parser (since it seems easy, but I  
understand that there are 2^1231234 possible variants).

I just updated my forum and I’m replying to this message by email to see  
what happens here on Meta. Thanks!

> Il 29 marzo 2016 alle 12.47 Régis Hanol [info@discourse.org](mailto:info@discourse.org) ha scritto:
> 
> > [@steko](#):
> >
> > but it seems to miss also the signature at the bottom of the message,
> 
> We **do not** remove user signatures.
> 
> Since there is no standards around user signatures, there is no **easy**  
> way to even get it to work for 75% of the emails.
> 
> The [best solution](https://github.com/mailgun/talon) I’ve found so far  
> requires machine learning and **lots** of training samples. It gets much  
> harder when you have to support more than one locale…
> 
> * * *
> 
> EDIT: I already added support for italian replies, but did push a new  
> version of the gem. I just did, you should update 😉
> 
> [update 'email\_reply\_trimmer' to latest version · discourse/discourse@480e8d0 · GitHub](https://github.com/discourse/discourse/commit/480e8d0dc7c5627b36ad3151719c67780fa3c5b6)
> 
> Posted by zogstrip on 03/29/2016
> 
> * * *
> 
> [Visit Topic](https://meta.discourse.org/t/email-reply-parsing/41597/8)  
> or reply to this email to respond
> 
> To stop receiving notifications for this particular topic, [click  
> here](https://meta.discourse.org/t/email-reply-parsing/41597/unsubscribe).  
> To unsubscribe from these emails, change your [user  
> preferences](https://meta.discourse.org/my/preferences)

Stefano Costa @stekosteko  
Editor, Journal of Open Archaeology Data

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [March 29, 2016, 12:54pm UTC](https://meta.discourse.org/t/email-reply-parsing/41597/10 "2016-03-29T12:54:28Z")

</div>

Due to a build failure, meta isn’t up to date yet…

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [March 30, 2016, 10:37am UTC](https://meta.discourse.org/t/email-reply-parsing/41597/11 "2016-03-30T10:37:05Z")

</div>

This topic was automatically closed after 24 hours. New replies are no longer allowed.
