But, since the email-in has Amazon’s `Message-ID` in the `In-Reply-To` and `References` headers, Discourse is unable to find the correct post.
A quick and dirty fix would seem to be to optionally add the wanted `Message-ID` to the `References` header on the emails Discourse sends out, with something like this added to [`lib/email/sender.rb`](https://github.com/discourse/discourse/blob/907f6cd76be55e6a9a0df8140921a4ce1a1d40d4/lib/email/sender.rb#L116):
```plaintext
if SiteSetting.message_id_stripped?
if post.post_number == 1
@message.header['References'] = topic_message_id
else
@message.header['References'] = [referenced_post_message_ids, topic_message_id, post_message_id].flatten.compact.uniq
end
end
```
This would then mean that the correct, Discourse-specified, `Message-ID` would be in the `References` header, and Discourse would be able to find the correct post to reply to.
However, I expect this would make whoever wrote the RFC cry a little. @zogstrip you seem to be the email master around here, what do you think?
---
### Author:  [@elijah](https://meta.discourse.org/u/elijah)
#### Post date: [May 10, 2017, 12:23am UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/6 "2017-05-10T00:23:03Z")
> [@LeoMcA](#):
>
> Yeah, I looked into this a bit more and it’s definitely because Amazon SES is replacing the Message-ID header with its own, like so:
Huh, they even document that:
> **[Amazon SES header fields - Amazon Simple Email Service](https://docs.aws.amazon.com/ses/latest/dg/header-fields.html)**
>
> Describes the header fields that Amazon SES accepts.
> **Note** : Amazon SES overrides any Message-ID header you provide.
I don’t think the RFCs _prohibit_ that behavior, but it is certainly not what I’d consider _expected_.
> [@LeoMcA](#):
>
> This would then mean that the correct, Discourse-specified, Message-ID would be in the References header, and Discourse would be able to find the correct post to reply to.
>
> However, I expect this would make whoever wrote the RFC cry a little. @zogstrip you seem to be the email master around here, what do you think?
It makes me cry a little, and I have to wonder what threading aware mail clients would do with that. If possible a verp-style variable Reply-To address would be much cleaner.
---
### Author:  [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [May 10, 2017, 12:23am UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/7 "2017-05-10T00:23:20Z")
I don’t suppose avoiding Amazon’s Spam Enablement Service is an option? Seriously, rewriting message IDs is head-desk-grade stupid.
---
### Author:  [@elijah](https://meta.discourse.org/u/elijah)
#### Post date: [May 10, 2017, 12:32am UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/8 "2017-05-10T00:32:39Z")
Some senders DKIM sign the message-id header, for example gmail:
```plaintext
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:from:date:message-id:subject:to;
bh=X5y5VSxfoLUD+lLIECebvv3q3y3QpWBKH7gRs5bnXag=;
b=y0xBqa28mfXxCYtJ0f4Lqv+e7Qe5rJU8/MSppGQXysTIm7dzZazLcpB7hsZ/UtfqPs
```
Checking it does not look like Discourse is among those senders:
```plaintext
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=discoursemail.com;
s=sjc1; t=1494307180;
bh=IqUdzg/T+JaQf72JvmWPvxOpfssOWEuOq0KRnDr12bU=;
h=Date:From:Reply-To:To:Subject:List-Unsubscribe;
b=uDaKehxZeEYu7YHrKNDZfeFP+iUylNwgyPATymo2LDpFkJrvUWq/gH5ZyW5zux6oG
```
I wonder if Amazon could be persuaded not muck with message-IDs if they were signed.
---
### Author:  [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [May 10, 2017, 12:45am UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/9 "2017-05-10T00:45:26Z")
Mail sent from us shouldn’t go through SES, so DKIM signing isn’t going to help people stuck using SES.
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [May 10, 2017, 3:19pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/10 "2017-05-10T15:19:36Z")
> [@mpalmer](#):
>
> I don’t suppose avoiding Amazon’s Spam Enablement Service is an option?
I’ve discussed it with my team, and we’re not immediately sure we have the budget for any other option. It’s cheap, but I guess there’s a reason why…
I also expect there’s a lot of other Discourse sites out there using SES, and it would be nice to come up with a fix for them too.
> [@elijah](#):
>
> If possible a verp-style variable Reply-To address would be much cleaner.
I’m not sure if this is what you were getting at, but it occurs to me that the reply key of each email is unique, and gets returned with every email-in. So if we were to (again, behind a pref, or conditionally on the SMTP creds being SES ones) on send store the reply key and associated post id in the db, and on receive fetch that post id from the db with the reply key, we’d be able to associate replies with their parents without making anyone at the IETF cry.
---
### Author:  [@elijah](https://meta.discourse.org/u/elijah)
#### Post date: [May 10, 2017, 5:28pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/11 "2017-05-10T17:28:47Z")
> [@LeoMcA](#):
>
> I’m not sure if this is what you were getting at, but it occurs to me that the reply key of each email is unique, and gets returned with every email-in. So if we were to (again, behind a pref, or conditionally on the SMTP creds being SES ones) on send store the reply key and associated post id in the db, and on receive fetch that post id from the db with the reply key, we’d be able to associate replies with their parents without making anyone at the IETF cry.
That is almost exactly what I meant. I just thought you could stick a `+topic/ID/POST` in the address, so you could just parse it out of there, if needed.
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [May 10, 2017, 6:56pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/12 "2017-05-10T18:56:57Z")
> [@LeoMcA](#):
>
> I’ve discussed it with my team, and we’re not immediately sure we have the budget for any other option. It’s cheap, but I guess there’s a reason why…
After doing some investigation into the prices of various services, this isn’t actually right. For the current numbers of emails we send (~60,000/month), SparkPost is _free_, and is cheaper than AWS up to ~120,000/month. Past that it still remains very competitive with AWS. Nothing else really comes close. [(All the data, and a slightly horrendous chart, can be found here.)](https://docs.google.com/spreadsheets/d/1fo39HsTr1ZSYB-k9AdgfgBNYjkzkoSsZc2NrJEy2KaE/edit?usp=sharing)
---
### Author:  [@DeanMarkTaylor](https://meta.discourse.org/u/DeanMarkTaylor)
#### Post date: [May 10, 2017, 7:33pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/13 "2017-05-10T19:33:52Z")
> [@LeoMcA](#):
>
> SparkPost is free
I highly recommend setting up this:
> [@Configure VERP to handle bouncing e-mails](https://meta.discourse.org/t/handling-bouncing-e-mails/45343):
>
> Setting up bounces handling Discourse uses the [Variable envelope return path (VERP)](https://en.wikipedia.org/wiki/Variable_envelope_return_path) technique to handle bouncing e-mails. To enable VERP, set the “reply by email address” site setting with an email address template that uses the + sign. The default of replies+%{reply\_key}@discourse.example.com … works fine. If you are using your own or your company’s email server, then, that’s it tada Your discourse instance must be able to receive email sent to this address. This should work via POP, bu…
Note that with all mail providers you will likely bump into deliverability issues, “free” services slightly more so - keep complaining at them if you do. Also consider breaking in a mail IP slowly if you do go that paid private IP route.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [May 10, 2017, 10:10pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/14 "2017-05-10T22:10:46Z")
That is fine, but I strongly recommend you pay for a dedicated IP in your mail services. Otherwise deliverability is a crapshoot.
---
### Author:  [@iva](https://meta.discourse.org/u/iva)
#### Post date: [June 17, 2017, 4:56pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/15 "2017-06-17T16:56:26Z")
Can anyone please clarify/confirm this for me: is it possible at all to use Amazon SES for emails with Discourse and have reply-by-email working correctly, or is it known not to work because of the Message-IDs being rewritten?
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [June 17, 2017, 9:30pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/16 "2017-06-17T21:30:36Z")
Unless Amazon has changed something, it looks like this has graduated to Known Issue, Don’t Do That.
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [June 19, 2017, 11:28am UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/17 "2017-06-19T11:28:06Z")
There was a lot of resistance to move away from SES, so I went about finding a technical solution.
As it turns out, it was incredibly simple, with a very well placed `return if`:
[https://github.com/discourse/discourse/pull/4938](https://github.com/discourse/discourse/pull/4938)
> [@riking](#):
>
> Unless Amazon has changed something, it looks like this has graduated to Known Issue, Don’t Do That.
How about graduating this to Known Issue, Toggle This Conveniently Placed Setting?
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [June 22, 2017, 2:17pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/19 "2017-06-22T14:17:07Z")
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [June 23, 2017, 4:00pm UTC](https://meta.discourse.org/t/email-in-replies-thread-wrongly/59365/20 "2017-06-23T16:00:00Z")
This topic was automatically closed after 25 hours. New replies are no longer allowed.