One of our users has posted feedback that emails are not being correctly threaded and believes this is because In-Reply-To is not correctly specified.
Currently Discourse will do this:
<topic/627@elixirforum.com>
But should it be something like:
<topic/627/4020@elixirforum.com>
(Which is what I think Github does)
Any thoughts on this please?
Here’s what we use when we send an email
post_identifier = "<topic/#{topic_id}/#{post_id}@#{host}>"
@message.header['Message-ID'] = post_identifier
This looks correct to me, we set the Message-Id header to the Id of the post. Your mail client should use that as the In-Reply-To header.
Thanks Régis, I have asked our member to register and post further info here if he continues to have problems.
Sorry for answering that late. Let me know if it’s necessary to create a new thread.
I am using Claws-mail client under Ubuntu GNU/Linux and so far I haven’t had this problem with other websites and mailing lists.
I will show how Google groups deals with headers “Message-id”, “In-Reply-To” and “References”
and contrast it to how Discourse does it.
1) Original Post
Date: Thu, 30 Jun 2016 21:49:11 -0700 (PDT)
Subject: [elixir-talk:13407] Workflow in mix / iex
Message-Id: <2efa495f-d3d0-44f0-9cf1-e029d8e2be94@googlegroups.com>
2) Answer to no. 1
Date: Fri, 1 Jul 2016 08:13:54 +0100
Subject: Re: [elixir-talk:13407] Workflow in mix / iex
Message-ID: <CAM-pwt51kB_6yn3YHbo6Wa5VyUDytqPhfzfRRTdSaqsLiYsLJg@mail.gmail.com>
In-Reply-To: <2efa495f-d3d0-44f0-9cf1-e029d8e2be94@googlegroups.com>
References: <2efa495f-d3d0-44f0-9cf1-e029d8e2be94@googlegroups.com>
3) Answer to no. 2
Date: Fri, 1 Jul 2016 06:10:35 -0700 (PDT)
Subject: Re: [elixir-talk:13408] Workflow in mix / iex
Message-Id: <2b3de36a-2a6c-45cb-a199-0fcf11bfae1b@googlegroups.com>
In-Reply-To: <CAM-pwt51kB_6yn3YHbo6Wa5VyUDytqPhfzfRRTdSaqsLiYsLJg@mail.gmail.com>
References: <2efa495f-d3d0-44f0-9cf1-e029d8e2be94@googlegroups.com>
<CAM-pwt51kB_6yn3YHbo6Wa5VyUDytqPhfzfRRTdSaqsLiYsLJg@mail.gmail.com>
and this is how Discourse does it
1) Original Post
Date: Sat, 17 Sep 2016 04:52:11 +0000
From: Paul <noreply@elixirforum.com>
Reply-To: Elixir Forum <mailserver+b4b32a98d201ecf58eef9e4c7510b978@elixirforum.com>
Message-ID: <topic/1689/10607@elixirforum.com>
In-Reply-To: <topic/1689@elixirforum.com>
References: <topic/1689@elixirforum.com>
Subject: [Elixir Forum] [Elixir Help] Semi-Manual backpressure
2) Answer to no. 1
Date: Sat, 17 Sep 2016 14:04:13 +0000
From: José Valim <noreply@elixirforum.com>
Reply-To: Elixir Forum <mailserver+81161440220741d3b07fff9c53460ba1@elixirforum.com>
Message-ID: <topic/1689/10618@elixirforum.com>
In-Reply-To: <topic/1689@elixirforum.com>
References: <topic/1689@elixirforum.com>
Subject: [Elixir Forum] [Elixir Help] GenStage Semi-Manual backpressure
3) Answer to no. 2
Date: Sat, 17 Sep 2016 14:59:21 +0000
From: José Valim <noreply@elixirforum.com>
Reply-To: Elixir Forum <mailserver+5a396d7011668a282e6465a58028ae8a@elixirforum.com>
Message-ID: <topic/1689/10623@elixirforum.com>
In-Reply-To: <topic/1689@elixirforum.com>
References: <topic/1689@elixirforum.com>
Subject: [Elixir Forum] [Elixir Help] GenStage Semi-Manual backpressure
And this is how I think it should be the Discourse version (changes in bold)
> 1) Original Post
> Date: Sat, 17 Sep 2016 04:52:11 +0000
> From: Paul <noreply@elixirforum.com>
> Reply-To: Elixir Forum <mailserver+b4b32a98d201ecf58eef9e4c7510b978@elixirforum.com>
> Message-ID: <topic/1689/10607@elixirforum.com>
> **In-Reply-To: [SHOULD NOT BE PRESENT]**
> **References: [SHOULD NOT BE PRESENT]**
> Subject: [Elixir Forum] [Elixir Help] Semi-Manual backpressure
> 2) Answer to no. 1
> Date: Sat, 17 Sep 2016 14:04:13 +0000
> From: José Valim <noreply@elixirforum.com>
> Reply-To: Elixir Forum <mailserver+81161440220741d3b07fff9c53460ba1@elixirforum.com>
> Message-ID: <topic/1689/10618@elixirforum.com>
> **In-Reply-To: <topic/1689/10607@elixirforum.com>**
> **References: <topic/1689/10607@elixirforum.com>**
> Subject: [Elixir Forum] [Elixir Help] GenStage Semi-Manual backpressure
> 3) Answer to no. 2
> Date: Sat, 17 Sep 2016 14:59:21 +0000
> From: José Valim <noreply@elixirforum.com>
> Reply-To: Elixir Forum <mailserver+5a396d7011668a282e6465a58028ae8a@elixirforum.com>
> Message-ID: <topic/1689/10623@elixirforum.com>
> **In-Reply-To: <topic/1689/10618@elixirforum.com>**
> **References: <topic/1689/10607@elixirforum.com>, <topic/1689/10618@elixirforum.com>**
> Subject: [Elixir Forum] [Elixir Help] GenStage Semi-Manual backpressure
based on the idea that <topic/1689@elixirforum.com> is an ID of a message that never existed (let’s call it “topic message id”)
I think there are few things causing trouble:
here’s a page that lists and references each message header.
https://www.cs.tut.fi/~jKorpela/headers.html
RFC 822
4.6. REFERENCE FIELDS
4.6. REFERENCE FIELDS
4.6.1. MESSAGE-ID / RESENT-MESSAGE-ID
This field contains a unique identifier (the local-part address unit) which refers to THIS version of THIS message. The uniqueness of the message identifier is guaranteed by the host which generates it. This identifier is intended to be machine readable and not necessarily meaningful to humans. A message identifier pertains to exactly one instantiation of a particular message; subsequent revisions to the message should each receive new message identifiers.
4.6.2. IN-REPLY-TO
The contents of this field identify previous correspondence which this message answers. Note that if message identifiers are used in this field, they must use the msg-id specification format.
4.6.3. REFERENCES
The contents of this field identify other correspondence which this message references. Note that if message identifiers are used, they must use the msg-id specification format.
RFC 1036
i cannot post more than 2 links so look for it in the first link
2.2.5. References
This field lists the Message-ID’s of any messages prompting the submission of this message. It is required for all follow-up messages, and forbidden when a new subject is raised. Implementations should provide a follow-up command, which allows a user to post a follow-up message. This command should generate a “Subject” line which is the same as the original message, except that if the original subject does not begin with "Re: " or "re: ", the four characters "Re: " are inserted before the subject. If there is no “References” line on the original header, the “References” line should contain the Message-ID of the original message (including the angle brackets). If the original message does have a “References” line, the follow-up message should have a “References” line containing the text of the original “References” line, a blank, and the Message-ID of the original message.
The purpose of the “References” header is to allow messages to be grouped into conversations by the user interface program. This allows conversations within a newsgroup to be kept together, and potentially users might shut off entire conversations without unsubscribing to a newsgroup. User interfaces need not make use of this header, but all automatically generated follow-ups should generate the “References” line for the benefit of systems that do use it, and manually generated follow-ups (e.g., typed in well after the original message has been printed by the machine) should be encouraged to include them as well.
It is permissible to not include the entire previous “References” line if it is too long. An attempt should be made to include a reasonable number of backwards references.
Looking forward to hearing from you.
All replies are replies to the parent topic, from Discourse’s perspective. A first post with lots of flat replies to that first post. One level of threading only, the topic. So I don’t think what you are proposing is correct.
So being that the case most of the issues are still valid, because you are using a message-id of a message that has never been posted.
This is how I think it should be the Discourse version (changes surrounded by **asterisks**)
1) Original Post
Date: Sat, 17 Sep 2016 04:52:11 +0000
From: Paul <noreply@elixirforum.com>
Reply-To: Elixir Forum <mailserver+b4b32a98d201ecf58eef9e4c7510b978@elixirforum.com>
Message-ID: <topic/1689/10607@elixirforum.com>
**In-Reply-To: [SHOULD NOT BE PRESENT]**
**References: [SHOULD NOT BE PRESENT]**
Subject: [Elixir Forum] [Elixir Help] Semi-Manual backpressure
2) Answer to no. 1
Date: Sat, 17 Sep 2016 14:04:13 +0000
From: José Valim <noreply@elixirforum.com>
Reply-To: Elixir Forum <mailserver+81161440220741d3b07fff9c53460ba1@elixirforum.com>
Message-ID: <topic/1689/10618@elixirforum.com>
**In-Reply-To: <topic/1689/10607@elixirforum.com>**
**References: <topic/1689/10607@elixirforum.com>**
Subject: [Elixir Forum] [Elixir Help] GenStage Semi-Manual backpressure
3) Answer to no. 1
Date: Sat, 17 Sep 2016 14:59:21 +0000
From: José Valim <noreply@elixirforum.com>
Reply-To: Elixir Forum <mailserver+5a396d7011668a282e6465a58028ae8a@elixirforum.com>
Message-ID: <topic/1689/10623@elixirforum.com>
**In-Reply-To: <topic/1689/10607@elixirforum.com>**
**References: <topic/1689/10607@elixirforum.com>**
Subject: [Elixir Forum] [Elixir Help] GenStage Semi-Manual backpressure
based on the idea that topic/1689@elixirforum.com is an ID of a message that never existed (let’s call it “topic message id”)
I think there are few things causing trouble:
Would you prefer In-Reply-To pointing to the previous post made, or everything pointing to the OP?
Keep in mind that a tree view is impossible. The best you’re going to get is a DAG, and even then someone can edit and quote a post below them (creating a loop and moving you to “general graph”).
@riking: I didn’t think of the complexities that would imply by letting people update their posts.
I think if the philosophy is to have a “flat tree”, then all replies to the original post should link to very first post id.
SettingIn-Reply-To to the last post made will make the email client display the conversation as a tree with as many levels as posts are in the thread.
So I guess implementing the changes I suggest in
thank you guys
Hi @riking ,
Could you describe how to create the situation where a post is referring to posts both above and below it? I did some stuffing around in our discourse install and couldn’t replicate that behaviour.
To answer your question: In-Reply-To should point to the previous post made IF the current post quotes it, or the OP if it doesn’t.
On this subject, I’ve seen [topic reply] @codinghorror comment that “All topic replies are conceptually “in reply to” the topic, though.” which he has repeated in [always topic] and recently in [flat replies] - its definitely the established view!
[topic reply]
[always topic]
[flat replies]
While true to an extent, when discourse is being used as a mailing list replacement the expectation that posts are linear is no longer true and it would be good for the headers to support that use case.
They almost do now!
thanks,
What you haven’t discussed is that a given Discourse reply can be “in reply to” three other posts upstream – anything you directly quote makes it a reply. Quote three posts, you are now in reply to those three posts.
You could only make a post in reply to downstream posts if you edited in the reply after the fact, which is also possible, but understandably rare and probably can be ignored.
[codinghorror] codinghorror
https://meta.discourse.org/users/codinghorror Jeff Atwood
https://meta.discourse.org/users/codinghorror co-founder
November 2What you haven’t discussed is that a given Discourse reply can be “in
reply to” three other posts upstream – anything you directly quote
makes it a reply. Quote three posts, you are now in reply to those three
posts.
Indeed, it would be a problem. In my mind multiple items were permitted
for the In-Reply-To header due to these parts of [RFC822].
4.6.2. IN-REPLY-TO
The contents of this field identify previous correspon-
dence which this message answers. Note that if message iden-
tifiers are used in this field, they must use the msg-id
specification format.
A.3.3. About as complex as you're going to get
[...]
In-Reply-To: <some.string@DBM.Group>, George's message
C.3.5. IN-REPLY-TO
The field-body is no longer a comma-separated list, although a
sequence is still permitted.
[RFC822]
https://www.ietf.org/rfc/rfc822.txt
Looking at [RFC2822] I see I was clearly wrong and only one is permitted.
3.6.4. Identification fields
The “Message-ID:” field contains a single unique message identifier.
The “References:” and “In-Reply-To:” field each contain one or more
unique message identifiers, optionally separated by CFWS.
The message identifier (msg-id) is similar in syntax to an angle-addr
construct without the internal CFWS.
message-id = “Message-ID:” msg-id CRLF
in-reply-to = “In-Reply-To:” 1*msg-id CRLF
[RFC2822]
https://www.ietf.org/rfc/rfc2822.txt
So while References could potentially be overloaded with the three posts
quoted (see next bit), In-Reply-To would have to choose one. The RFC
then goes on (further down 3.6.4) to say
Note: Some implementations parse the “References:” field to display
the “thread of the discussion”. These implementations assume that
each new message is a reply to a single parent and hence that they
can walk backwards through the “References:” field to find the parent
of each message listed there. Therefore, trying to form a
“References:” field for a reply that has multiple parents is
discouraged and how to do so is not defined in this document.
So even using References falls in the ‘undefined behaviour’ category.
That I did not expect.
You could only make a post in reply to downstream posts if you edited in
the reply after the fact, which is also possible, but understandably
rare and probably can be ignored.
Given what I’ve put above (which was new to me, I must have missed it
during my first reading of the relevant documents), I could only suggest
making In-Reply-To point to the most recent post quoted when sending an
email which (to my mind) provides the closest mapping to intuitive email
behaviour.
Thanks for taking the time to go over this (again) with me and I hope
the suggestion above seems reasonable if not ideal.
thanks,
kk
I recently pushed a fix that will:
In-Reply-To” and “References” headers when it’s a reply (ie. post_number > 1)In-Reply-To” header to the newest of the referenced (either replied to or quoted) postsReferences” header to the topic as well as all referenced posts (if any).This should ensure the emails are threaded properly on any decent 
client 
Thanks @zogstrip!
I can’t speak for @astonj or @eksperimental but the fix was noticed by users in my community!
Hopefully this helps in Emails are not threaded in Outlook 2013 too!