Emails are not threaded in Outlook 2013

Getting back to the subject of the topic: can anyone confirm that, with changes since June, email notifications from Discourse are properly threaded in Outlook 2013?

1 Like

Trying to reply to this post to see what happens…

(EDIT: ok, I forgot to turn on email notifications wnen visiting the site recently for meta.discourse.org before doing this, apparently. So instead of spamming this thread twice more, I’ll just subscribe to the currently-most-active thread and report back.)

I can confirm that the emails are still not properly threaded in Outlook 2013 :frowning:

New gist at Discourse headers for 2014-10-13 · GitHub

Still no List-Post header, although who knows if that’s the important one.

I wasn’t originally subscribed to the topics I just subscribed to, so it’s possible if you guys are using the “In-Reply-To the top post” strategy (instead of the “In-Reply-To most recent post on the thread”) threading won’t work unless you receive the email for the first post. I’ve stayed subscribed to all of meta.discourse.org so if that happens to be the case we should know after a few posts.

I wonder if the "Re: " subject prefix is important!?

No idea… someone with knowledge of Outlook 2013 internals will need to step up, otherwise it’ll just be the way it is.

From your gist, here are the current relevant headers that I know of:

Subject: [Discourse Meta] [feature] Possibility to selectively (or completely) mark notifications as read
From: Mittineague <info@discourse.org>
Reply-To: Discourse Meta <reply+29fb8d02de845c19eb575be161de8855@discourse.org>
In-Reply-To: <topic/20227@meta.discourse.org>
References: <topic/20227@meta.discourse.org>
Auto-Submitted: auto-generated
List-ID: <feature.meta.discourse.org>
List-Archive: https://meta.discourse.org/t/possibility-to-selectively-or-completely-mark-notifications-as-read/20227
List-Unsubscribe: <https://meta.discourse.org/my/preferences>
Precedence: list

One thing I did notice , comparing your GitHub email header gists:

Reply-To: felixge/node-sandboxed-module
<reply+i-21866655-ede66648bf2cdb8a3707a09c0e03d118d318ad1e-617481@reply.github.com>

versus

Reply-To: Discourse Meta <reply+29fb8d02de845c19eb575be161de8855@discourse.org>

@techapj can you improve us here so we are using topic/20227 instead of Discourse Meta here as the “friendly” reply-to name here, like GitHub does?

Couple other worries about magical incantations needed:

  • Re: in the email title? So ghetto, but we do now support it (off by default).
  • Message-ID: might need to be something other than the random generic technobabble string our mail server seems to slap on there. Is it safe for us to include this header @michael?

Where is that setting? I can’t find it in user preferences at least, and although I am an admin for discourse.specifiction.org, I don’t see any real adjustable settings there.

Hi @domenic!

Update discourse.specifiction.org to the latest version, then see:

Happy to report that since upgrading to the latest Discourse and turning on the optional Re: prefix, the emails are now threading.

2 Likes

Well, it does not really work in every client:

Roundcube:

KMail:

Thunderbird:

Thunderbird seems to grab the "Re: " prefix and the other clients don’t. As the In-Reply-To headers are wrong these clients could not do any threading. As mentioned before the specs are here (chapter 3.6.4): http://www.ietf.org/rfc/rfc2822.txt

I have edited the In-Reply-To and References Headers manually according to the specs. I have replaced these topic links with the appropriate message IDs. Suprise, surprise, it threads well in every client:

KMail:

Roundcube:

@codinghorror: Have a look at rfc2822 and @peternlewis post above. That’s the real issue with its intended solution. Adding an Re: prefix is just a workaround for some clients.

in-reply-to is set to this value

In-Reply-To: <topic/16501@meta.discourse.org>

Thus, all replies should thread to this topic, if what you say is true. Is it?

No. You may talk about these headers:

In-Reply-To: <topic/16501@meta.discourse.org>
References: <topic/16501@meta.discourse.org>

It has been explained here why it is wrong:

In-Reply-To and References must point to a unique message id. <topic/16501@meta.discourse.org> is not a unique message id thus mail clients cannot do threading. It works when I replace these headers with its correct message ids manually on the imap server.

edit:
I had a look on a github email. They do it this way:

Message-ID: <discourse/onebox/pull/257/r20941543@github.com>
In-Reply-To: <discourse/onebox/pull/257@github.com>
References: <discourse/onebox/pull/257@github.com>

Maybe fixing the Message-ID in discourse mails suffices. Dicourse does it this way (wrong!):

Message-ID: <5475a885c34fd_2fd5036a24480ae@archvm.mail>
In-Reply-To: <topic/753@discourse.sevenbyte.org>
References: <topic/753@discourse.sevenbyte.org>

Hmmm… How about this?

Message-ID: <topic/16501/27@meta.discourse.org>
In-Reply-To: <topic/16501/26@meta.discourse.org>
References: <topic/16501/25@meta.discourse.org> <topic/16501/26@meta.discourse.org>

What you describe is impossible – I don’t know how to get, in Ruby code, the message ID header that the mail server appends to emails at the time it sends them, before dispatching the email. That ID doesn’t even exist until the message is being transmitted.

Those ids are created at the time the message is sent by the mail server as far as I know, and they are internal to the mail server software.

Also, the “in reply to” should always be in reply to the original topic creation, the first post of the topic, which you may never have even received an notification about. So there is no “message id” to work with in that case…

As far as I know it is created by the first SMTP server when it is missing. Maybe this could help; it should be possible with action mailer.

That is not that bad. I just tried it on a mailman mailinglist in my inbox. I deleted the parent message and the threading did not break because the References headers are fine. Maybe also this commit description here could help a bit: https://projects.archlinux.org/aur.git/commit/?id=8658bf22aac73ed8d32d3f998c3c13528f415ffe

see here:

   Note: Some implementations parse the "References:" field to display
   the "thread of the discussion".  These implementations assume that
   each new message is a reply to a single parent and hence that they
   can walk backwards through the "References:" field to find the parent
   of each message listed there.

As @codinghorror mentioned in-reply-to must be the first email. So it should be sth. like this, should’t it?

Message-ID: <topic/16501/27@meta.discourse.org>
In-Reply-To: <topic/16501/1@meta.discourse.org>
References: <topic/16501/25@meta.discourse.org> <topic/16501/26@meta.discourse.org>

Which is fine, but let’s say this happens:

  1. You read a topic.
  2. You reply to post #25 in that topic as post #27.
  3. You go offline for hours.
  4. Post #28 arrives and mentions your name (it is not a reply)

So there are three numbers in play here:

  1. the topic (post #1)
  2. Your post (post #27)
  3. The post which mentions you (post #28)

Note that this post is a reply to the topic. That’s the typical case, we want email to thread based on the TOPIC. We have no idea if you have an email about the first post, though.

So when you get an email notification about the mention (post #28), that email should contain:

Message-ID: <topic/16501/28@meta.discourse.org>
In-Reply-To: <topic/16501@meta.discourse.org>
References: <topic/16501@meta.discourse.org> 

I guess “References” is the key here? All the posts in a topic should “reference” the first post, even though you may not have that email message.

Right now it says

Message-ID: <54752ba335286_633fee205cc454883a2@tiefighter6-virtual_host.mail>
In-Reply-To: <topic/91@talk.thumbtack.io>
References: <topic/91@talk.thumbtack.io>

So the only thing missing is the Message-ID header. All replies are in reply to the first post of the topic, though…

OK I just checked this in

topic_identifier = "<topic/#{topic_id}@#{host}>"
post_identifier = "<topic/#{topic_id}/#{post_id}@#{host}>"
@message.header['Message-ID'] = post_identifier
@message.header['In-Reply-To'] = topic_identifier
@message.header['References'] = topic_identifier

Let’s see if that helps? no idea, but I can check headers…

3 Likes

Also here’s what a random GitHub reply has in it

Message-ID: <discourse/discourse/pull/3006/c64666078@github.com>
In-Reply-To: <discourse/discourse/pull/3006@github.com>
References: <discourse/discourse/pull/3006@github.com>

so this might be the missing header we needed, if it works.

1 Like

OK I can confirm this works. In a reply I just got via email here I see

Message-ID: <topic/22572/86235@meta.discourse.org>
In-Reply-To: <topic/22572@meta.discourse.org>
References: <topic/22572@meta.discourse.org>

So it corresponds pretty exactly with what GitHub sends in their email notifications. Thanks for the nudge @rumpelsepp hopefully this helps.

1 Like

You’re welcome. :blush:
It looks fine now but I don’t know if there are any issues left. I will just test it a few days. In case of issues I will ping you back. Anyway, thanks for the fix!

1 Like

Threads now well in KMail, K9 and Thunderbird! But only Roundcube has still problems with this. Maybe this is now a roundcube upstream issue rather than a discourse one, isn’t it?

2 Likes