Email reply parsing

I’ve finally setup reply by email for my Discourse forum, but unfortunately parsing of replies seems suboptimal and the original message is not stripped. My forum is in Italian, but that should be accounted for, at least judging by some earlier bug reports about this same issue. I’ve seen a few other topics here on Meta (and there are probably older ones as well):

I don’t use Gmail, that probably accounts for 50% of my users alone and should be covered, but I wonder if there’s a more robust way to support quirks of other email clients (e.g. mobile email clients, since replying to email from mobile is one of the use cases that Discourse wants to support, I think). Of course I can just suggest users to strip entirely the original message or leave only short quotes if they like, but … I’m sure Discourse can handle this better!

We need specific examples of failures, PM full source of emails to @zogstrip

1 Like

Ok, will do. As a test, this is a reply from a webmail client.

Il 26 marzo 2016 alle 13.16 Sam Saffron info@discourse.org ha scritto:

We need specific examples of failures, PM full source of emails to
@zogstrip

Posted by sam on 03/26/2016


Visit Topic
or reply to this email to respond

To stop receiving notifications for this particular topic, click
here
.
To unsubscribe from these emails, change your user
preferences

Stefano Costa @stekosteko
Editor, Journal of Open Archaeology Data

2 Likes

This might be unrelated but…

Make sure you are on the latest version.

There was a bug apparently fixed last week where the "<details>" section was not being removed, that contained the previous posts.

I’m currently experiencing this one, but I haven’t updated to see if the fix does resolve the issue.

2 Likes

Yes, I saw the <details>-related bug but I upgrade daily to the latest version so I don’t think it affected me when testing reply via email today. Thanks for pointing this out however!

3 Likes

The Italian dates seem to be tripping the email parser, we need a smarter heuristic

1 Like

Today I tried removing the original message from a reply entirely, but it seems to miss also the signature at the bottom of the message, it’s kept as is (but that shouldn’t be language-dependent) :confused:

We do not remove user signatures.

Since there is no standards around user signatures, there is no easy way to even get it to work for 75% of the emails.

The best solution I’ve found so far requires machine learning and lots of training samples. It gets much harder when you have to support more than one locale…


EDIT: I already added support for italian replies, but did push a new version of the gem. I just did, you should update :wink:

https://github.com/discourse/discourse/commit/480e8d0dc7c5627b36ad3151719c67780fa3c5b6

1 Like

Oh, I was under the impression that signatures starting with “–\n” or “–
\n” were already removed by the parser (since it seems easy, but I
understand that there are 2^1231234 possible variants).

I just updated my forum and I’m replying to this message by email to see
what happens here on Meta. Thanks!

Il 29 marzo 2016 alle 12.47 Régis Hanol info@discourse.org ha scritto:

We do not remove user signatures.

Since there is no standards around user signatures, there is no easy
way to even get it to work for 75% of the emails.

The best solution I’ve found so far
requires machine learning and lots of training samples. It gets much
harder when you have to support more than one locale…


EDIT: I already added support for italian replies, but did push a new
version of the gem. I just did, you should update :wink:

https://github.com/discourse/discourse/commit/480e8d0dc7c5627b36ad3151719c67780fa3c5b6

Posted by zogstrip on 03/29/2016


Visit Topic
or reply to this email to respond

To stop receiving notifications for this particular topic, click
here
.
To unsubscribe from these emails, change your user
preferences

Stefano Costa @stekosteko
Editor, Journal of Open Archaeology Data

Due to a build failure, meta isn’t up to date yet…

This topic was automatically closed after 24 hours. New replies are no longer allowed.