Importing posts with Markdown

Migration
1

I rewrote https://github.com/discourse/discourse/blob/main/script/import_scripts/json_generic.rb a bit for my own data. It works well but I cannot figure out how to have Discourse use Markdown in the imported posts.

For example a post might be
**some text here in bold** and something else here
After importing Discourse shows it verbatim like that, not using the ** Markdown syntax to bold the “some text here in bold” part. And weirdly also if I try to edit such a post using the normal Discourse GUI, it will ignore any formatting.

The import script uses
cook_method: Post.cook_methods[:raw_html],
I also tried
cook_method: Post.cook_methods[:regular],
without any visible change.

How can I make Discourse recognise Markdown when importing data?

1 Like
2

Even if you add formatted text at the end of the post, this text won’t be formatted?

Does Rebuild HTML do something?

1 Like
3

Yes, it’s super weird. The preview looks correct but if I save the edit, the unformatted post is shown.

Rebuild HTML does not fix it.

4

What does the raw look like in the database, can you check it with data explorer? Or output in the rails console.

Can you ensure that the rebaking job is executed in Sidekiq when you trigger Rebuild HTML?

Or rebake with the rails console to see if there’s some error?

Those are just random suggestions :person_shrugging: It looks like the post isn’t cooked.

1 Like
5

Thank you!

Using SELECT raw=cooked FROM posts WHERE ... returns true for the post.

I have no idea what that means or how I can check that :smiley: Could you give me a pointer?

I ran bundle exec rake posts:rebake and the post is the same as before and behaves the same as before on editing.

6

Forget about that, that won’t help after all.

I’d create a new post in which I’d paste the same markdown content as one of the unformatted posts.

I’d expect the new post to be properly formatted by Discourse.

Then, I’d compare the raw field of both posts; either with data explorer or the rails console.

If one is formatted when cooked, and the other one isn’t when you rebuild HTML (which rebakes the post), there should be a difference in their raw. :thinking:

7

What happens if you edit the post? Does it look like you expect? Does it render correctly if your edit and save?

8

I took one of the offending posts, went into its edit mode, copied the contents and made a new posts with it. The new posts uses the Markdown formatting, I see bold text and a link with a title.

In the database their raw is identical but their cooked is not.

SELECT (SELECT raw FROM posts WHERE id = 13846) = (SELECT raw FROM posts WHERE id = 13735); → t

SELECT (SELECT cooked FROM posts WHERE id = 13846) = (SELECT cooked FROM posts WHERE id = 13735); → f

Clicking “Rebuild HTML” on the badly formatted posts does no change.

/logs/ shows nothing at this time (only older warnings and errors).

9

When I edit it, the preview looks correct, the Markdown formatting is applied there (e.g. “foo”). Once I save, the post will still show the formatting characters (e.g. “**foo**”)

10

Can you try running

Post.update_all(cook_method: Post.cook_methods[:regular])

in the Rails console and then “Rebuild HTML” again?

Note that it will update all your posts to use the regular cooking method.

1 Like
12 
/var/www/discourse$ bundle exec rails c
Loading production environment (Rails 7.0.7)
irb(main):001:0> Post.update_all(cook_method: Post.cook_methods[:regular])
=> 13809
irb(main):002:0>

Done! Sadly no change.

But I solved it. And I am sorry for not mentioning what plugins I had installed as they can interfere…

I have the Topic Ratings Plugin installed and enabled for the category into which my imports go.

On a whim I disabled the plugin and tried again to edit a “broken” post. Now upon saving, the Markdown formatting was used for fancy rendering. I re-enabled the plugin and tried again with success. I imported new posts and it also renders the posts correctly now (using cook_method: Post.cook_methods[:regular]).

I have no idea what is/was going on there but it seems to have been fixed by toggling the Topic Ratings Plugin off and on again.

Thank you all for the great suggestions and problem hunting!

1 Like
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.