# メーリングリストモード：メール内の「アップロード」リンクが壊れている？
**URL:** https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354
**Category:** Bug
**Created:** [2019 年 3 月 22 日午後 5:50 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354 "2019-03-22T17:50:38Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@distfp](https://meta.discourse.org/u/distfp)
#### Post date: [2019 年 3 月 22 日午後 5:50 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/1 "2019-03-22T17:50:38Z")
Hello,
I’m subscribed to the Rust forums, which use Discourse, and after first submitting this bug there, I was suggested to post it here:
On the Rust Discourse forums I have “Mailing list mode” enabled in my preferences, and also “Include previous replies at the bottom of emails”.
In the plain-text (non-HTML) version of such e-mails, the replies for a thread (e.g. [Ideas for a Rayon logo - #17 - community - The Rust Programming Language Forum](https://users.rust-lang.org/t/ideas-for-a-rayon-logo/26323/17)) have links to images of the form `upload://...` which means it’s not possible to view them.
In the HTML version (as well as in the web UI) they have the form `https://discourse-cdn-sjc1.com/business5/uploads/...` It would be nice to have these full URLs in the text version too.
Thanks
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 23 日午前 5:18 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/2 "2019-03-23T05:18:04Z")
Yes, this is definitely a bug if we are leaking raw URLs into the text content of HTML emails.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 24 日午後 11:45 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/3 "2019-03-24T23:45:46Z")
This is a very interesting problem.
At the moment we simply ship the `raw` markdown to the `plaintext` section of emails.
We made that decision many years ago and were kind of aware of some of the cracks.
@eviltrout / @codinghorror how do you feel about changing the mailer so it does `HTML` =\> `text` and uses that for the text part of the MIME vs using Post `raw`?
We now have a very robust patterns for doing `HTML` =\> `text` living inside Discourse and could make a custom one here. It is a pretty big job though cause we would have to be ultra careful with quotes and oneboxes.
For example now text contents has stuff like:
```plaintext
[quote="codinghorror, post:2, topic:112354"]
this is definitely a bug
[/quote]
```
This would probably be a nicer way of rendering this in a text email.
```plaintext
> this is definitely a bug
URL-TO-QUOTE
```
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 25 日午前 4:01 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/4 "2019-03-25T04:01:21Z")
Well, Markdown is designed to look good in raw text.. it’s one of the design directives
> Readability, however, is emphasized above all else. A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions.
Problem is we have nasty BBcode in there. I’d rather convert BBCode to text, personally, rather than playing a game of “telephone” where we go Markdown → HTML → text
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 25 日午前 4:16 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/5 "2019-03-25T04:16:21Z")
> [@codinghorror](#):
>
> I’d rather convert BBCode to text, personally, rather than playing a game of “telephone” where we go Markdown → HTML → text
We would need to design Markdown → Markdown renderer. We could teach markdown.it to do this with a heavily modified pipeline, it is not trivial, probably a couple of weeks for a first shot at it. Even if we got uploads and bbcode hacks we have working we would still be left with inline HTML tags.
Playing “telephone” here is faster and more robust, cause we get to normalize the Markdown that way.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 25 日午前 4:30 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/6 "2019-03-25T04:30:05Z")
> [@sam](#):
>
> We would need to design Markdown -\> Markdown renderer.
No, we only need
BBCode → Plain Text
You don’t need to touch the existing Markdown, just convert BBCode
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 25 日午前 4:36 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/7 "2019-03-25T04:36:39Z")
> [@codinghorror](#):
>
> You don’t need to touch the existing Markdown, just convert BBCode
It is not that simple:
```plaintext
[quote="codinghorror, post:6, topic:112354"]
You don’t need to touch the existing Markdown, just convert BBCode
[/quote]
```
Is BBCODE
```plaintext
`[b]test[/b]`
```
Is not BBCODE cause it is ` quoted
In the past we went down the regex hell of hack on top of hack to allow for all the edge cases and it we ended up with a nightmare pipeline.
The problem is there is no clean way of fishing out BBCODE and only converting it.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 25 日午前 4:46 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/8 "2019-03-25T04:46:23Z")
I’m not really following, playing telephone and going
Markdown → HTML → text
is nowhere near clean, as HTML doesn’t convert to text properly, there are zillions of edge cases in that, plus “telephone” adds multiple layers of errors in the conversion.
I still think it makes more sense to **surgically convert BBCode to plain text** , because the Markdown is _guaranteed_ to look OK as plain text. It’s literally in the definition of the Markdown project:
> Readability, however, is emphasized above all else. A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions.
The problem is the BBCode, so I think it makes the most sense to focus on the problem rather than going for 3 levels of conversions on everything, even Markdown that **does not need to be converted to “plain text”**.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 25 日午前 4:51 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/9 "2019-03-25T04:51:32Z")
Say I have a post with this Markdown:
```plaintext
I just tried typing `[b]bold[/b]` and I did not get bold.
```
If we blindly convert this we get:
```plaintext
I just tried typing ` **bold** ` and I did not get bold.
```
This is clearly wrong, so we add a regex, and then another regex for ``` to handle hoisting, and then for 3 space indents and finally have a regex tower high enough that it is both impossible to maintain and somehow works for all the crazy cases people hit in the while.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 25 日午前 5:15 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/10 "2019-03-25T05:15:10Z")
Right but isn’t that part of the normal Markdown pipeline? You have to do that anyway, so at the time the markdown pipeline is _already_ deciding to hoist stuff out, hoist it all out, and then convert BBCode to Markdown.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 25 日午前 5:19 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/11 "2019-03-25T05:19:29Z")
Yes we can do this.
> [@sam](#):
>
> We would need to design Markdown -\> Markdown renderer. We could teach markdown.it to do this with a heavily modified pipeline, it is not trivial, probably a couple of weeks for a first shot at it
But it is pretty complex change, certainly not a 1 day change, I think my estimate of 2 weeks is probably about right here.
HTML-\>Markdown is a matter of hours of work especially since @vinothkannans already built this for our quote and cut-and-paste feature.
---
### Author:  [@distfp](https://meta.discourse.org/u/distfp)
#### Post date: [2019 年 3 月 25 日午前 6:44 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/12 "2019-03-25T06:44:31Z")
> At the moment we simply ship the `raw` markdown to the `plaintext`
> section of emails.
Having markdown in the body of e-mails is nice (some mail clients may
prettify it), there are just a few kinks with the current format, as you
mention.
> For example now text contents has stuff like:
>
> ```plaintext
> [quote="codinghorror, post:2, topic:112354"]
> this is definitely a bug
> [/quote]
>
> ```
>
> This would probably be a nicer way of rendering this in a text email.
>
> ```plaintext
> > this is definitely a bug
> URL-TO-QUOTE
>
> ```
This is reason why I wrote my “Discourse article” extension to Emacs’
Gnus mail client: the transformation of the Discourse quote format into
regular e-mail citation format.
And then I added other things, and realized that except for a handful of
transformations (including a header for the “Previous Replies” section
in mailing list mode and separators for replies therein) I could hand
off most of the rendering to markdown mode…
I would definitely welcome a more “canonical” markdown, if you will.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 28 日午前 12:29 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/13 "2019-03-28T00:29:41Z")
Something about this still does not sit right with me @sam – we are letting ugly BBCode style markup leak into the raw.
I guess I can see the HTML → text as a quick fix, but to me it does not address the underlying issue.
Perhaps we just don’t care that much about this, the plain text version of the email is a “best effort” thing so if we have @vinothkannans existing code to do it, that would be good enough?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 28 日午前 12:34 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/14 "2019-03-28T00:34:19Z")
> [@codinghorror](#):
>
> so if we have @vinothkannans existing code to do it, that would be good enough?
I think it may be worth trying it out, on the upside this can lead to other fixed like… what happens when you quote a onebox and reply? what happens when you quote a poll and reply.
It is replacing one evil with a different evil, but it may be crazy enough to work… not sure.
* * *
About to quote ⬇
Let’s do a little test here…
_Poll ([view on site](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/14))_
A quote:
> [@codinghorror](#):
>
> so if we have @vinothkannans existing code to do it, that would be good enough?
A Bold bold
```plaintext
A [b]Bold[/b] bold
```
A onebox:
https://embed.reddit.com/r/i3wm/comments/49pwag/changing_the_title_bar_font/?embed=true&ref_source=embed&ref=share
A mini onebox: [Reddit - Please wait for verification](https://www.reddit.com/r/i3wm/comments/49pwag/changing_the_title_bar_font/)
An emoji 🙂
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 28 日午前 12:35 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/15 "2019-03-28T00:35:41Z")
````plaintext
About to quote :arrow_down:
Let’s do a little test here…
A quote:
> so if we have @vinothkannans existing code to do it, that would be good enough?
A Bold bold
```
A [b]Bold[/b] bold
```
A onebox:
 [reddit](https://www.reddit.com/r/i3wm/comments/49pwag/changing_the_title_bar_font/)
### [r/i3wm - Changing the title bar font](https://www.reddit.com/r/i3wm/comments/49pwag/changing_the_title_bar_font/)
0 votes and 5 comments so far on Reddit
A mini onebox: [Changing the title bar font : i3wm](https://www.reddit.com/r/i3wm/comments/49pwag/changing_the_title_bar_font/)
An emoji :slight_smile:
````
We totally EAT polls, oneboxes are odd, so I am not sure how great this is as a solution.
Maybe we just have to pay the price and do this properly in a custom renderer.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [2019 年 3 月 28 日午前 12:37 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/16 "2019-03-28T00:37:34Z")
I still think “strip all bbcode” is NOT as hard as you’re making it out to be.
Most bbcode could just be stripped with no real loss of meaning, and only a handful would need conversion.
Because the Markdown is plain text compatible as-is ..
---
### Author:  [@distfp](https://meta.discourse.org/u/distfp)
#### Post date: [2019 年 3 月 28 日午後 12:09 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/17 "2019-03-28T12:09:37Z")
I see improving plain text e-mails could prove quite complicated!
From my point of view, the readability of the plain text e-mails is already quite good and indeed, not much conversion is required. Here’s what I think could improve readability the most, in order of decreasing importance:
- Links to images (and other assets?): currently unusable (see my
original post) as `upload://` links can’t be handled by mail
clients.
- Quotes: my naive conversion into regular e-mail citation improved
readability _a lot_, so having the server do it would be great.
- Oneboxes, as you mentioned – although I don’t think I’ve seen them
often.
- Polls: I don’t think polls would be very useful in plain text as there
would be no means of voting. Showing the results when the poll is
closed could be nice, but in any case I think a simple link to the
Discourse post would do, polls being infrequent.
- Previous Replies: if possible, this section could be in markdown form
too (section heading, post separators…).
I understand this could be a lot of effort, and possibly not worth it if
there aren’t may users reading the e-mails in plain text – which I’m
guessing is the case.
I’d say that fixing the `upload://` links would have the most impact on
usability, based on all the Discourse posts I’ve received by mail.
Could this be fixed independently of more involved pipeline changes?
(And thank you for having plain text in the first place 👍🏻)
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 3 月 29 日午前 4:51 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/18 "2019-03-29T04:51:50Z")
Thanks Damien,
I have filed this in my mind castle 🏰 , and set a reminder for next week, we may have to just make some small hacky attempts here vs a fully 100% robust super implementation, will decide what path we are going to take here some time next week. At a minimum something for uploads cause that is by far the worst pain we have here.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2019 年 11 月 26 日午前 7:38 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/20 "2019-11-26T07:38:36Z")
数ヶ月経ってもまだ私のリストに残っており、これはまたもや、逆マーキングダウンパーサーが本当に必要になるような、小さくても非常に厄介なエッジケースの一例です。
この問題の根本は、この画像の生データが：
となっていることです。
ここで考えられる簡単な（ただし少し破壊的で、コードブロックの内容を壊してしまう）方法は、以下のように置換することです：
`upload://bkLTU01hzKturAl7WsxEXeHYk1y.png`
を
`https://global.discourse-cdn.com/meta/original/3X/4/f/4f710992a5de8a73bfc0198ac9fa557cdfcc3380.png`
に置き換える。
単純な正規表現の gsub を使えば、`0.01%` のようなケースでは明らかに様々な問題を引き起こしてしまいますが、このスレッドの OP にとっての `99.9%` のケースは改善されるでしょう。
（また、不正な投稿が 10,000 回のクエリを発生させないように注意する必要があるので、何らかのバッチ処理で行う必要があります）
@codinghorror、この方針で問題ありませんか？正直少し抵抗がありますが、逆パーサーがない現状ではこれが最善の策に近いと思います。
cc @cvx / @zogstrip
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [2019 年 11 月 26 日午前 11:15 UTC](https://meta.discourse.org/t/mailing-list-mode-upload-links-broken-in-e-mails/112354/21 "2019-11-26T11:15:38Z")
> [@sam](#):
>
> 単純な正規表現の gsub を使えば、`0.01%` のケースでは明らかに様々な問題を引き起こしますが、ここでは OP の `99.9%` のケースを改善できます。
それは実行可能で、安全性もそれなりにありそうです。
@cvx、それをあなたのリストに追加してくれませんか？メールを送信する前に、生データや Markdown を加工して、すべての `upload://` URL を絶対 URL に変更する必要があります。