# Maintain tables from HTML email in when using HTML to Markdown
**URL:** https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434
**Category:** Feature
**Created:** [6월 13, 2017, 4:27오후 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434 "2017-06-13T16:27:48Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@mcwumbly](https://meta.discourse.org/u/mcwumbly)
#### Post date: [6월 13, 2017, 4:27오후 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/1 "2017-06-13T16:27:48Z")
> [@Email reply includes inline styles with "incoming email prefer html" site setting on](https://meta.discourse.org/t/email-reply-includes-inline-styles-with-incoming-email-prefer-html-site-setting-on/61639/2):
>
> Now that we have a [Html To Markdown converter](https://github.com/discourse/discourse/blob/master/lib/html_to_markdown.rb)
I was trying to advise a user on how they might import a Google Doc into Discourse as a new topic and suggested trying to copy/paste the doc into an email and then email it in. It mostly worked, but there were a few tables in the doc that got mangled in the process.
When an email into Discourse includes an HTML table, that table gets mangled and turned into plaintext on the Discourse side.
I have the `incoming_email_prefer_html` on as well as the “allow html tables” setting.
It’d be cool if tables were just skipped and maintained as HTML in that case. (Or perhaps there’s a table extension we’re planning on enabling once the markdown-it is in the mix?)
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [5월 3, 2018, 8:53오전 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/3 "2018-05-03T08:53:50Z")
This _might_ work better now that we have more formal Markdown table support @vinothkannans?
---
### Author:  [@vinothkannans](https://meta.discourse.org/u/vinothkannans)
#### Post date: [5월 3, 2018, 9:06오전 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/4 "2018-05-03T09:06:44Z")
> [@mcwumbly](#):
>
> I was trying to advise a user on how they might import a Google Doc into Discourse as a new topic and suggested trying to copy/paste the doc into an email and then email it in.
Yes, it will work better **if** the user used copy-paste method instead of email-in workaround.
HTML pasting feature uses client side JavaScript for Markdown conversion. But for incoming emails we converting it in server side using our [custom converter](https://github.com/discourse/discourse/blob/master/lib/html_to_markdown.rb) which doesn’t support Markdown tables yet.
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [5월 3, 2018, 2:45오후 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/5 "2018-05-03T14:45:11Z")
I see no point in duplicating our code in JS and Ruby. I will remove the Ruby version and replace it with the JS version.
---
### Author:  [@brechtm](https://meta.discourse.org/u/brechtm)
#### Post date: [2월 6, 2020, 3:34오후 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/6 "2020-02-06T15:34:12Z")
우연히 표가 포함된 이메일을 받는데, 첫 번째 표와 그 뒤에 오는 내용은 마크다운으로 변환되는 과정에서 사라집니다. 이로 인해 메시지가 갑자기 끊기는 현상이 발생하여 사용자들에게 매우 혼란을 줍니다. 작은 봉투 아이콘을 클릭하면 원본 HTML 이메일을 읽을 수 있지만, 이는 사용자 친화적이지 않으며 버그로 간주되어야 할 것입니다.
메시지를 편집하여 원시 HTML 표를 그대로 붙여넣으면 잘 작동합니다. 마크다운 변환 과정에서도 이렇게 하면 안 될까요?
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [4월 29, 2020, 3:35오후 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/7 "2020-04-29T15:35:45Z")
이 PR이 병합되면 테이블이 올바르게 변환됩니다 👌
>
>
> TLDR; this commit vastly improves how whitespaces are handled when converting fr…om HTML to Markdown.
> It also adds support for converting HTML \`\\` to markdown tables.
>
> \---
>
> The previous \`remove\_whitespaces!\` method was traversing the whole HTML tree and used a heuristic to remove
> leading and trailing whitespaces whenever it was appropriate (ie. mostly before and after HTML block elements)
>
> It was a good idea, but it was very limited and leaded to bad conversion when the html had leading whitespaces on several lines for example.
> One such example can be found \[here\](https://meta.discourse.org/t/86782).
>
> For various reasons, most of the whitespaces in a HTML file is ignored when the page is being displayed in a browser.
> The rules that the browsers follow are the \[CSS' White Space Processing Rules\](https://www.w3.org/TR/css-text-3/#white-space-rules).
> They can be quite complicated when you take into account RTL languages and other various tidbits but they boils down to the following:
>
> \- Collapse whitespaces down to one space (0x20) inside an inline context (ie. nodes/tags that are being displaying on the same line)
> \- Remove any leading/trailing whitespaces inside an inline context
>
> One quick & dirty way of getting this 90% solved would be to do \`HTML.gsub!(/\[\[:space:\]\]+/, " ")\`.
> We would also need to hoist \`\\` elements in order to not mess with their whitespaces.
> Unfortunately, this solution let some whitespaces creep around HTML tags which leads to more \`.strip!\` calls than I can bear.
>
> I decided to "\_emulate\_" the browser's handling of whitespaces and came up with a solution in 4 parts
>
> \#### 1. \`remove\_not\_allowed!\`
>
> The HtmlToMarkdown library is recursively "visiting" all the nodes in the HTML in order to convert them to Markdown.
> All the nodes that aren't handled by the library (eg. \`\\`, \`\\` or any non-textual HTML tags) are "swallowed".
> In order to reduce the number of nodes visited, the method \`remove\_not\_allowed!\` will automatically delete all the nodes
> that have no "visitor" (eg. a \`visit\_\\` method) defined.
>
> \#### 2. \`remove\_hidden!\`
>
> Similar purpose as the previous method (eg. reducing number of nodes visited), there's no point trying to convert something that is hidden.
> The \`remove\_hidden!\` method removes any nodes that was hidden using the "hidden" HTML attribute, some CSS or with a width or height equal to 0.
>
> \#### 3. \`hoist\_line\_breaks!\`
>
> The \`hoist\_line\_breaks!\` method is there to handle \`\\` tags. I know those tiny \`\\` don't do much but they can be quite annoying.
> The \`\\` tags are inline elements but they visually work like a block element (ie. they create a new line).
> If you have the following HTML "\`\Foo\Bar\\`", it ends up visually similar to "\`\Foo\\\Bar\\`".
> The latter being much more easy to process than the former, so that's what this method is doing.
> The \`hoist\_line\_breaks\` will hoist \`\\` tags out of inline tags until their parent is a block element.
>
> \#### 4. \`remove\_whitespaces!\`
>
> The \`remove\_whitespaces!\` is where all the whitespace removal is happening. It's broken down into 4 methods as well
>
> \- \`remove\_whitespaces!\`
> \- \`is\_inline?\`
> \- \`collapse\_spaces!\`
> \- \`remove\_trailing\_space!\`
>
> The \`remove\_whitespace!\` method is recursively walking the HTML tree (skipping \`\\` tags).
> If a node has any children, they will be chunked into groups of inline elements vs block elements.
> For each chunks of inline elements, it will call the \`collapse\_space!\` and \`remove\_trailing\_space!\` methods.
> For each chunks of block elements, it will call \`remote\_whitespace!\` to keep walking the HTML tree recursively.
>
> The \`is\_inline?\` method determines whether a node is part of a inline context.
> A node is inline iif it's a text node or it's an inline tag, but not \`\\`, and all its children are also inline.
>
> The \`collapse\_spaces!\` method will collapse any kind of (white) space into a single space (" ") character, even across tags.
> For example, if we have "\` Foo \\n\ Bar \\\t42\`", it will return "\`Foo \Bar \42\`".
>
> Finally, the \`remove\_trailing\_space!\` method is there to remove any trailing space that might creep in at the end of the inline chunk.
>
> This solution is not 100% bullet-proof.
> It does not support RTL languages at all and has some caveats that I felt were not worth the work to get properly fixed.
>
> \---
>
> FIX: switched Nokogiri to Nokogumbo for better HTML5 parsing
> FIX: better detection of hidden elements when converting HTML to Markdown
> FIX: take into account the \`allowed\_href\_schemes\` site setting when converting HTML \`\\` to Markdown
> FIX: added support for 'mailto:' scheme when converting \`\\` from HTML to Markdown
> FIX: added support for \`\\` dimensions when converting from HTML to Markdown
> FIX: added support for \`\\`, \`\\` and \`\\` when converting from HTML to Markdown
> FIX: added support for multilines emphases, strongs and strikes when converting from HTML to Markdown
> FIX: added support for \`\\` when converting from HTML to Markdown
> DEV: remove unused 'sanitize' gem
>
> Wow, did you just read all that?! Congratz, here's a cookie: 🍪.
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [4월 30, 2020, 10:22오전 UTC](https://meta.discourse.org/t/maintain-tables-from-html-email-in-when-using-html-to-markdown/64434/8 "2020-04-30T10:22:39Z")