CommonMark testing started here!

Per our migration plans, I just enabled markdown.it at Meta.

It is an excellent CommonMark compliant engine. I expect quite a few edges to be found during this period of testing.

Do not open bugs about the new Engine, just reply here.

I/we will be updating the top post here with the list of issues and striking out stuff as it is fixed.

Current known issues (to be fixed)

  • :white_check_mark: Onebox vanishes after you start typing and works not-super consistently

  • :white_check_mark: pending approval does not render

  • :white_check_mark: Quote without preceding empty line does not work.

  • :white_check_mark: Onebox of reply is missing tons of info and not rendering right

  • :white_check_mark: When quoting the composer sometimes inserts a quote block in the middle of a line (not a new bug but broken with new engine)

  • :white_check_mark: Onebox bypass is not working

  • :white_check_mark: Table is not showing up styled in preview

  • :white_check_mark: Details composer toolbar generating inline details

  • :white_check_mark: Emoji → Image conversion (when posing Emoji’s via a phone) is not implemented yet.

  • :white_check_mark: 2 polls in a post seems to be … not that good.

  • :white_check_mark: Sanitizer stripping out right align style for tables

  • :white_check_mark: Some BBCode is unimplemented: [email], [url], [code] and [img]

  • :white_check_mark: Somelink - just another link on the internet :arrow_backward: fails cause of the tag/category plugin

  • :white_check_mark: Mention after paragraph is strange

  • Can add any oneboxes after an IMG tag

  • :email, class_name: is not an emoji

  • https://medium.com/@nicksantos not autolinked

New features

Issues that will not be fixed

  • Mixing inline and multiline BBCode will no longer be supported.

  • Core will not implement [ul] [ol] and [li] support for BBCode cause it is a recipe for failure.

This topic is wiki, if you are TL3 or above feel free to clean it up.

31 Likes
Discourse CommonMark migration plans :confetti_ball: :balloon:
Emphasis links do not work
Only one heading in Markdown quote?
Text missing following fenced code block
Discourse Mangling Text in a Code Block
Preformatted text should show what I pasted
Markdown isn't parsing/displaying some content properly
Details markup not working correctly in messages
'undefined' appearing in quotes inside [quote]s
The Markdown parser fails to process the [[**text**]](url) expression
BBCode tags don't work accross multiple 'blocks'
Dropcaps in Discourse - cannot override span tag
Mini (Inline) Onebox Support RFC
Spoiler with text on the same line as the tag no longer works
Discourse Math Plugin
Is it necessary to rebake all posts during major version releases?
Importing / migrating from vBulletin 4
Importing / migrating from vBulletin 4
Spoilers not working for external images
Text disappearing between breakline/heading and quote
Unexpected removal of text in markdown conversion
Embedding backticks twice collapses whitespace
Markdown link not parsed when prepended with `[Text] `
Fenced codeblocks with extra spaces at the end don't work?
Unable to create poll when the last poll option starts with hashtag
Link consumes first character that follows
Bold and Italic in Chinese
Display of images in links break if they contain parenthesis
Post text does not render (firstBlock.shift is not a function)
Newlines at the end of list items should not yield <br>s
Markdown quote requires blank line to end quote
\\\1 In Code Block Fails
Post editor; Heading not working for less than 3 charactors
Censored words out of a URL
A comma followed by an underscore in a URL results in defunct link
Discourse doesn't follow the CommonMark spec for HTML <pre> tags inside code spans
Preformatted text not allowing characters
Indenting of code blocks is incorrect when close to text
In bulleted list of mentioned tags, last tag is not formatted correctly as a link
Improve numbered lists
URL auto-linking doesn't hyperlink if open paren precedes the protocol
Odd markdown results - unexpected whitespace removal

Wikipedia Table (converted to markdown)

Name Programming language License
bbPress PHP[2] GPL[3]
Beehive Forum PHP GPL
Discourse Ruby, JavaScript GPL
Discuz! PHP Discuz! EULA
eXo Platform Java LGPL
FluxBB PHP GPL
FUDforum PHP GPL
Ikonboard Perl Proprietary
Invision Power Board PHP Proprietary
MyBB PHP LGPL, moving to BSD in 2.0[14]
Phorum PHP Phorum License 2 (BSD like)
PHPWind PHP PHPWind EULA
phpBB PHP GPL
PunBB PHP GPL
Simple Machines PHP BSD License
Syndie Java Open source, multiple licenses
UBB.threads PHP Proprietary
Vanilla Forums PHP Proprietary/GPL[24]
vBulletin PHP[27] Proprietary[28]
Thredded Ruby MIT

It’s working :tada:

25 Likes

Nesting code fences demo:

Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```

And here’s how it’s coded:

````markdown
Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```
````

To nest code blocks with a ``` codefence, just keep making outer codefences longer.

Will it nest to arbitrary depths? Six backtick version:

`````text
````markdown
Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```
````
`````

And SEVEN!

``````text
`````text
````markdown
Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```
````
`````
``````

This is the feature I’ve wanted for quoting things in “how to post in Discourse” messages.

7 Likes

I have difficult when I need to insert some code in a post.
It seems that [code][/code] and <pre></pre> no longer work.

Will it be permanent or simply have not been implemented yet?

Let’s see

<pre><code>int main(int argc, char** argv) {
return 0;
</code></pre>
int main(int argc, char** argv) {
return 0;

On my forum if I insert <pre><code> </code></pre> I 've this result:

and this is the result when I use only <pre></pre> or <code></code>

So when commonMark will be fully implemented we need to use only <pre><code> </code></pre> ?

1 Like

Fixed a bunch of edge cases with quoting and bbcode blocks

quote after a list

  • list
Fixed a bunch of edge cases with quoting and bbcode blocks
[quote]
nested
[quote]
test
[/quote]
[/quote]

quote after a list
- list 
[quote]
test
[/quote]

https://github.com/discourse/discourse/commit/0e0092d73837f69c0592083bea5a19516871f1a8

10 Likes

Nothing really changed with <pre> or <code> both work exactly as they used to …

I am pre-formatted 
   text
<pre>
I am pre-formatted 
   text
</pre>

It used to work and continues to work.

1 Like

Is there a way to disable a onebox and just show a link? Previously leading whitespace did that, but now that doesn’t seem to work.

vv two spaces
   https://github.com/discourse/discourse/tree/master/app/mailers

Shows as

https://github.com/discourse/discourse/tree/master/app/mailers

I did manage to disable with the ugly:

 https://github.com/discourse/discourse/tree/master/app/mailers &nbsp;

https://github.com/discourse/discourse/tree/master/app/mailers  

But that’s not so easy to type.

8 Likes

It is a bug :ant: will fix

It may take a day or so cause I need some guidance on the best approach to fix it from @Vitaly

https://github.com/markdown-it/markdown-it/issues/375

5 Likes

Seems i did not understood your question on github, sorry. Now got what you asked about.

I see 2 possible workarounds “in spirit of markdown”

  1. Define link text explicit (because such links should not be autoreplaced).
    • Less convenient.
  2. Use <http://...> autolink (wrap with <...>)
    • Easy, but user may wish to use it in very rage case when linkifier fails, to define link borders right. In this case such link will not be oneboxed.

Let me know if that is acceptable. Implementation should be simple - just test more link_open token attributes.

4 Likes

Trouble is that users have already been trained for a very long time that

spacehttp://somelink.com does not get a onebox.

and

http://somelink.com does get a onebox.

It is going to be super painful to teach everyone another option here.

2 Likes

I understand :(. Sometime we should do horrible things, because users are not going to change their habits.

Then try patched paragraph rule (which stores info about trimmed head), as discussed on github. The only disadvantage is - it may be changed in upstream on CM spec update. But you can add test to signal about such change.

3 Likes
1 2 3 4
Call 11 12 11 13
Text 15 1 5 18
Browse 122 2 6 110

I wish it showed properly in the preview so I do not have to post this as a topic.

1 Like

That is a bug, we will get it fixed!

8 Likes

The oneboxing issue is fixed:

https://github.com/discourse/discourse/commit/0d62420cbe443b2cf1061405f83f34eef5103532

https://github.com/discourse/discourse/commit/0d62420cbe443b2cf1061405f83f34eef5103532

 https://github.com/discourse/discourse/commit/0d62420cbe443b2cf1061405f83f34eef5103532

I did want to take a few moments to say :sun_with_face: :dancer: :sunny: to @Vitaly, it is absolutely amazing one can replace core behavior of the new engine in a very tidy and maintainable way without needing to fork the project. I am able to achieve in minutes stuff that used to take me hours when working with the previous Markdown engines.

The pipeline, date flow and token stream make for an amazingly clear design that is understandable by humans. Thank you.

14 Likes

I enabled the typographer here (with the site setting enable_markdown_typographer)

This means that:

(c) (C) (r) (R) (tm) (TM) (p) (P) +-

test.. test... test..... test?..... test!....

!!!!!! ???? ,,  -- ---

"Smartypants, double quotes" and 'single quotes'

renders as:

© © ® ® ™ ™ § § ±

test… test… test… test?.. test!..

!!! ??? , – —

“Smartypants, double quotes” and ‘single quotes’

I do not think we will enable it by default, but some communities have expressed interest in it so … here you go…

13 Likes

I fixed that per:

https://github.com/discourse/discourse/commit/fed57be9edd73c31842626176e4a68da4f5f3f43

Should be deployed here shortly.

4 Likes