Why does markdown do this? (whitespace trimming and other features)

[Rhababo]

Apologies for the vague title,

I’m in the process of modifying some of the functionality with markdown on discourse to better suit the needs of my community which is coming over from a mybb forum.

Discourse is my first exposure to markdown, and I want to take the approach of “don’t tear down a fence if you don’t know why it’s there.” However, I am having difficulty finding out why markdown takes certain actions and I would love some resources to help me understand, (especially if there are some security concerns I could be overlooking).

The response in this topic provided some good general references including some early discussion and a (now defunct) plugin:

Rich Text Formatting (WYSIWYG)

Hi all, The users of my Discourse community are sports teachers who are not very technical, therefore the enforced use of markdown in the editor has created some formatting problems in posts. Ideally, we could enable a rich text WYSIWYG editor to make posting as simple and familiar as possible. Are there any ways to enable this or, if not, plans for inclusion on the roadmap? Cheers

And this post has been incredibly helpful on the development side of things:

Developer's guide to Markdown extensions

Discourse uses a Markdown engine called Markdown-it. Here are some dev notes that will help you either fix bugs in core or create your new plugins. The Basics Discourse only contains a few helpers on top of the engine, so the vast majority of learning that needs to be done, is understanding Markdown It. The docs directory contains the current documentation. I strongly recommend reading: The architecture document to understand at a top level how the engine works. Development for basic…

But, again, I’m looking to understand why certain features exist - so i can best evaluate removing or tweaking them.

The current features in my crosshairs are:

1. Removing consecutive line breaks without </br> or other code.
2. Automatic creation of code blocks on lines with 4 or more leading spaces.
3. Trimming leading whitespace on a new line.
4. Turning any sequence of numbers into a numbered list in consecutive increasing order.

Any guidance is appreciated! Thanks!

[Canapin (Coin-coin le Canapin)]

I’m very interested by this topic but I don’t have the knowledge to answer most of your questions.

However,

@Rhababo

Removing consecutive line breaks without </br> or other code.

@Rhababo

Trimming leading whitespace on a new line.

I think Discourse wants to keep posts clean and easy to read.
By removing extra line breaks and trimming spaces at the beginning of lines, it ensures posts look consistent.
This helps especially when users, sometimes even without realizing (I know some!), add random spaces or lines which could make their content harder to read.

@Rhababo

Turning any sequence of numbers into a numbered list in consecutive increasing order.

It may look weird to force this, especially when you actually want a gap in your list, but my guess is that it’s to keep numbered lists as simple as possible, with no need to re-number every element when you add a new one to your post (or in a wiki post).

[Jagster (Jakke Lehtonen)]

@Canapin

lines which could make their content harder to read

Actually keeping everything really thight can make things harder to read. Paragraph after an image can be an example.

But… I’m using <br /> so no biggie at all.

Another but… I’m quite sure that cleaning operation has nothing to do with cleaner reading experience. It comes from code itself. Similar thing than very tired statement that markdown is easier to read on ”code level”. End user is never on that situation.

Well. This is pure meta now.

But I refuse to believe cleaning lines and extra spaces is actually making overall experience better to end user.

[simon]

@Rhababo

Removing consecutive line breaks without </br> or other code.

I’m not sure if this will solve a problem for you, but Discourse has a traditional markdown linebreaks site setting that is disabled by default. When enabled, two trailing spaces are required to create a line break.

Here’s an example with that setting enabled:

Here’s an example with the setting disabled (the default value of the setting):

@Rhababo

Discourse is my first exposure to markdown,

It might be worth looking at this page and trying its 10 minute markdown tutorial: Markdown Reference. The Discourse new user tutorial links to that page, but it likely gets overlooked by a lot of users who take the tutorial:

[Heliosurge (Dan DeMontmorency)]

@Jagster

But I refuse to believe cleaning lines and extra spaces is actually making overall experience better to end user.

Definitely makes ascii art a challenge to display properly.

[Rhababo]

@Canapin

I think Discourse wants to keep posts clean and easy to read.
By removing extra line breaks and trimming spaces at the beginning of lines, it ensures posts look consistent.

@Canapin

but my guess is that it’s to keep numbered lists as simple as possible

I really appreciate the insight. My community has a large forum game/storytelling sub-community where freedom in formatting tends to be worth more than consistency. If the core reason for these features is really to help maintain a consistent and simplified look, it is probably safe to ax them for my use case, or to at least give them a user selectable toggle.

@simon

It might be worth looking at this page and trying its 10 minute markdown tutorial: Markdown Reference .

This is a very helpful tutorial, and its emphasis on “making beautiful text” does seem to follow the ideas of consistency mentioned earlier. Thanks for pointing it out.

[Rhababo]

In doing some extra reading about markdown I have found this site particularly enlightening.

This is, I think, the original description of markdown, as cited by the commonmark team, and it reveals that a core element of why markdown does what it does is due to its proximity to html.

For instance the first sentence there is:

Markdown is a text-to-HTML conversion tool for web writers.

And I think that some of these quirks of markdown may be less due to their organizational properties and more to a desire to cleanly convert the text to corresponding html code.

This explains why whitespace is trimmed, because html trims its whitespace.

Also the reason for turning any sequence of numbers into an increasing numbered list becomes a little clearer with:

https://daringfireball.net/projects/markdown/syntax#list

@https://daringfireball.net/projects/markdown/syntax#list

It’s important to note that the actual numbers you use to mark the list have no effect on the HTML output Markdown produces. The HTML Markdown produces from the above list is:

<ol>
<li>Bird</li>
<li>McHale</li>
<li>Parish</li>
</ol>

If you instead wrote the list in Markdown like this:

1.  Bird
1.  McHale
1.  Parish

or even:

3. Bird
1. McHale
8. Parish

you’d get the exact same HTML output. The point is, if you want to, you can use ordinal numbers in your ordered Markdown lists, so that the numbers in your source match the numbers in your published HTML. But if you want to be lazy, you don’t have to.

So markdown doesn’t even bother looking at the numbers, because ordered lists in html don’t bother assigning numbers. (The actual code in markdown-it does check the character in the first position, but only to start the numbering from that character).

I think there may be more to it beyond this, (such as formatting consistency across multiple devices) but some of the quirkier aspects do seem very html inspired.