More regarding Discourse and "MailMate"

More on the issue of blank invites, just received:

MailMate will be handling it in the next update. But the point is that Discourse is producing an illegal character set descriptor according the MIME spec. en_US is a language, not a character set, and it makes no sense in this context. Sure, everyone has to handle it, but that doesn’t mean it shouldn’t be fixed, and if a mail program can’t parse that, it’s going to default to us-ascii rather than utf-8, because that’s the default for that content-type.

(MailMate is a markdown-based, extremely customizable, Mac IMAP client.)

1 Like

As this appears to be the first report in five or six years it looks like MailMate was the outlier in handling this practice, rather than Discourse?

I haven’t heard of MailMate before. But the invitee was almost lost due to the problem. The related MIME specs are fairly explicit, and most mail systems are relatively lax about such issues on the ingress side, but it’s still worthy of note in terms of conformance.

I have removed the dupe topic on this:

The email has an incorrect header:

Content-Type: text/plain; charset=“en_US.UTF-8”; format=flowed
The charset should just be utf-8 or nothing at all for this specific email.

Specifically this

Content-Type: text/plain; charset="en_US.UTF-8"; format=flowed

should instead be

Content-Type: text/plain; charset="UTF-8"; format=flowed

is that correct?

1 Like

I tested this today and can’t see the erroneous charset in the raw message.

2 Likes

I’m unsure where to look here to find the raw Discourse messages to inspect those headers directly. I’ll note another mail-related issue in passing. Some of the emails generated by Discourse use newline sequences (\n), others use carriage-return/newline sequences (\r\n). I’ve noticed this with various generated emails from Discourse. For example, I have two topic summary emails here from my own forum that were generated one after another during the day. The first uses \n, the second uses \r\n. They otherwise look like the same format. I discovered this when some users reported seeing extraneous ^M characters in the displayed email.

You can view the source (including headers) of any email from most mail clients. Gmail exposes it natively too over the web.

Well yeah. As it happens, I’ve been writing MUAs and MTAs since ARPANET days. However, I’d like to examine Discourse email before its ingress into the associated MTA intrinsic to Discourse. Since some summaries use \r\n and some use only \n, there’s another factor obviously involved in there somewhere.

Weird flex, but ok.

There are numerous ways to do this, and you can redirect SMTP easily on a test instance. If you’ve been writing mail transfer agents for somewhere between 30 and 50 years I’m surprised you don’t have any debug/troubleshooting tools in your arsenal. Suffice to say anything we can offer here is going to fall woefully short of your experience.

Presumably the specific problem at hand is occurring during the assembly of the messages by Discourse before entering the actual SMTP flow – I don’t see any obvious way that my own production SMTP environment (that Discourse is using) would add carriage return chars to some summaries and not others. But given that I am not an expert at Discourse internals, and really don’t have the time to become one at present, I’d hope that someone with more familiarity with the summary message assembly functions in Discourse could more readily identify the fork in the road, so to speak.

Actually, I can see the same effect in the last two notifications sent to me via Discourse (both to the same dest email address, of course). The one that starts “You can view the source” doesn’t have any extraneous carriage return chars. The next one, which was my reply on the thread, has an extra carriage return after every line in the message, including the "Visit Topic: and “To unsubscribe” lines.

Carriage return normalization should be easy to fix provided we can narrow down where it is good/correct and wrong/bad in emails specifically.

2 Likes