CommonMark testing started here!


(Sam Saffron) #1

Per our migration plans, I just enabled markdown.it at Meta.

It is an excellent CommonMark compliant engine. I expect quite a few edges to be found during this period of testing.

Do not open bugs about the new Engine, just reply here.

I/we will be updating the top post here with the list of issues and striking out stuff as it is fixed.

Current known issues (to be fixed)

  • :white_check_mark: Onebox vanishes after you start typing and works not-super consistently

  • :white_check_mark: pending approval does not render

  • :white_check_mark: Quote without preceding empty line does not work.

  • :white_check_mark: Onebox of reply is missing tons of info and not rendering right

  • :white_check_mark: When quoting the composer sometimes inserts a quote block in the middle of a line (not a new bug but broken with new engine)

  • :white_check_mark: Onebox bypass is not working

  • :white_check_mark: Table is not showing up styled in preview

  • :white_check_mark: Details composer toolbar generating inline details

  • :white_check_mark: Emoji -> Image conversion (when posing Emoji’s via a phone) is not implemented yet.

  • :white_check_mark: 2 polls in a post seems to be … not that good.

  • :white_check_mark: Sanitizer stripping out right align style for tables

  • :white_check_mark: Some BBCode is unimplemented: [email], [url], [code] and [img]

  • :white_check_mark: Somelink - just another link on the internet :arrow_backward: fails cause of the tag/category plugin

  • :white_check_mark: Mention after paragraph is strange

  • Can add any oneboxes after an IMG tag

  • :email, class_name: is not an emoji

  • Nick Santos – Medium not autolinked

New features

Issues that will not be fixed

  • Mixing inline and multiline BBCode will no longer be supported.

  • Core will not implement [ul] [ol] and [li] support for BBCode cause it is a recipe for failure.

This topic is wiki, if you are TL3 or above feel free to clean it up.


Discourse CommonMark migration plans :confetti_ball: :balloon:
Emphasis links do not work
Ordered list mis-numbering
Text missing following fenced code block
Only one heading in Markdown quote?
Underscore in URL link causes markdown problem
Markdown isn't parsing/displaying some content properly
Details markup not working correctly in messages
'undefined' appearing in quotes inside [quote]s
The Markdown parser fails to process the [[**text**]](url) expression
BBCode tags don't work accross multiple 'blocks'
Dropcaps in Discourse - cannot override span tag
Mini (Inline) Onebox Support RFC
Mathjax plugin install instructions?
Spoilers not working for external images
Add image in table
Discourse Math Plugin
Spoiler with text on the same line as the tag no longer works
Text disappearing between breakline/heading and quote
Unexpected removal of text in markdown conversion
Embedding backticks twice collapses whitespace
Markdown link not parsed when prepended with `[Text] `
Fenced codeblocks with extra spaces at the end don't work?
Unable to create poll when the last poll option starts with hashtag
Link consumes first character that follows
Bold and Italic in Chinese
Display of images in links break if they contain parenthesis
Post text does not render (firstBlock.shift is not a function)
Newlines at the end of list items should not yield <br>s
Intra-word Markdown italic and bold not supported?
Markdown quote requires blank line to end quote
\\\1 In Code Block Fails
Post editor; Heading not working for less than 3 charactors
Censored words out of a URL
A comma followed by an underscore in a URL results in defunct link
Discourse doesn't follow the CommonMark spec for HTML <pre> tags inside code spans
Preformatted text not allowing characters
Indenting of code blocks is incorrect when close to text
In bulleted list of mentioned tags, last tag is not formatted correctly as a link
Improve numbered lists
URL auto-linking doesn't hyperlink if open paren precedes the protocol
Odd markdown results - unexpected whitespace removal
Discourse Mangling Text in a Code Block
Preformatted text should show what I pasted
(Sam Saffron) #2

Problems with Markdown rendering: Links that contains apostrophes
(Rafael dos Santos Silva) #6

Wikipedia Table (converted to markdown)

Name Programming language License
bbPress PHP[2] GPL[3]
Beehive Forum PHP GPL
Discourse Ruby, JavaScript GPL
Discuz! PHP Discuz! EULA
eXo Platform Java LGPL
FluxBB PHP GPL
FUDforum PHP GPL
Ikonboard Perl Proprietary
Invision Power Board PHP Proprietary
MyBB PHP LGPL, moving to BSD in 2.0[14]
Phorum PHP Phorum License 2 (BSD like)
PHPWind PHP PHPWind EULA
phpBB PHP GPL
PunBB PHP GPL
Simple Machines PHP BSD License
Syndie Java Open source, multiple licenses
UBB.threads PHP Proprietary
Vanilla Forums PHP Proprietary/GPL[24]
vBulletin PHP[27] Proprietary[28]
Thredded Ruby MIT

It’s working :tada:


(Eli the Bearded) #7

Nesting code fences demo:

Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```

And here’s how it’s coded:

````markdown
Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```
````

To nest code blocks with a ``` codefence, just keep making outer codefences longer.

Will it nest to arbitrary depths? Six backtick version:

`````text
````markdown
Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```
````
`````

And SEVEN!

``````text
`````text
````markdown
Here's how to make a code block in markdown:

```cpp
int main(int argc, char** argv) {
return 0;
```
````
`````
``````

This is the feature I’ve wanted for quoting things in “how to post in Discourse” messages.


(Daniela) #8

I have difficult when I need to insert some code in a post.
It seems that [code][/code] and <pre></pre> no longer work.

Will it be permanent or simply have not been implemented yet?


(Jeff Atwood) #9

Let’s see

<pre><code>int main(int argc, char** argv) {
return 0;
</code></pre>
int main(int argc, char** argv) {
return 0;

(Daniela) #10

On my forum if I insert <pre><code> </code></pre> I 've this result:

and this is the result when I use only <pre></pre> or <code></code>

So when commonMark will be fully implemented we need to use only <pre><code> </code></pre> ?


(Sam Saffron) #15

Fixed a bunch of edge cases with quoting and bbcode blocks

quote after a list

  • list
Fixed a bunch of edge cases with quoting and bbcode blocks
[quote]
nested
[quote]
test
[/quote]
[/quote]

quote after a list
- list 
[quote]
test
[/quote]

(Sam Saffron) #17

Nothing really changed with <pre> or <code> both work exactly as they used to …

I am pre-formatted 
   text
<pre>
I am pre-formatted 
   text
</pre>

It used to work and continues to work.


(Eli the Bearded) #18

Is there a way to disable a onebox and just show a link? Previously leading whitespace did that, but now that doesn’t seem to work.

vv two spaces
   https://github.com/discourse/discourse/tree/master/app/mailers

Shows as

discourse/app/mailers at master · discourse/discourse · GitHub

I did manage to disable with the ugly:

 https://github.com/discourse/discourse/tree/master/app/mailers &nbsp;

discourse/app/mailers at master · discourse/discourse · GitHub  

But that’s not so easy to type.


(Sam Saffron) #19

It is a bug :ant: will fix

It may take a day or so cause I need some guidance on the best approach to fix it from @Vitaly


#20

Seems i did not understood your question on github, sorry. Now got what you asked about.

I see 2 possible workarounds “in spirit of markdown”

  1. Define link text explicit (because such links should not be autoreplaced).
    • Less convenient.
  2. Use <http://...> autolink (wrap with <...>)
    • Easy, but user may wish to use it in very rage case when linkifier fails, to define link borders right. In this case such link will not be oneboxed.

Let me know if that is acceptable. Implementation should be simple - just test more link_open token attributes.


(Sam Saffron) #21

Trouble is that users have already been trained for a very long time that

spacehttp://somelink.com does not get a onebox.

and

http://somelink.com does get a onebox.

It is going to be super painful to teach everyone another option here.


#22

I understand :(. Sometime we should do horrible things, because users are not going to change their habits.

Then try patched paragraph rule (which stores info about trimmed head), as discussed on github. The only disadvantage is - it may be changed in upstream on CM spec update. But you can add test to signal about such change.


(Republic Wireless) #23
1 2 3 4
Call 11 12 11 13
Text 15 1 5 18
Browse 122 2 6 110

(Republic Wireless) #24

I wish it showed properly in the preview so I do not have to post this as a topic.


(Sam Saffron) #25

That is a bug, we will get it fixed!


(Sam Saffron) #26

The oneboxing issue is fixed:

Space prior to onebox stops onebox resolution · discourse/discourse@0d62420 · GitHub

 https://github.com/discourse/discourse/commit/0d62420cbe443b2cf1061405f83f34eef5103532

I did want to take a few moments to say :sun_with_face: :dancer: :sunny: to @Vitaly, it is absolutely amazing one can replace core behavior of the new engine in a very tidy and maintainable way without needing to fork the project. I am able to achieve in minutes stuff that used to take me hours when working with the previous Markdown engines.

The pipeline, date flow and token stream make for an amazingly clear design that is understandable by humans. Thank you.


(Sam Saffron) #27

I enabled the typographer here (with the site setting enable_markdown_typographer)

This means that:

(c) (C) (r) (R) (tm) (TM) (p) (P) +-

test.. test... test..... test?..... test!....

!!!!!! ???? ,,  -- ---

"Smartypants, double quotes" and 'single quotes'

renders as:

© © ® ® ™ ™ § § ±

test… test… test… test?.. test!..

!!! ??? , – —

“Smartypants, double quotes” and ‘single quotes’

I do not think we will enable it by default, but some communities have expressed interest in it so … here you go…


Autocorrect quote marks outside of code blocks
(Sam Saffron) #28

I fixed that per:

Should be deployed here shortly.