How to repair old titles?

(Markus) #1

Hello guys,
I‘m not sure, if this issue was solved before?

I have a lot of old posts / wiki articles using titles like …


As you might know, a few versions earlier this year, someone changed the behavior how to interpret these titles. Currently, the output isn’t one of these shiny markdown titles with bold text …


… instead, it‘s just …


How can I automatically fix this issue for all old articles and add the missing space between the hashtags and title text?

It becomes very frustrating to do this manually.

Any clue? Thanks in advance!

Posts with ##Headlines broken after 1.8->1.9 upgrade
(Gerhard Schlager) #2

I just pushed a fix that should allow you to do this:

rake posts:remap['^(#+)([^#\\s].*)$','\\1 \\2','regex']

(Markus) #3

Thanks for your quick response.

Unfortunately, It didn’t worked for me. First I‘ve updated Discourse to the latest version via rebuild. Secondly, I‘ve executed the rake task.

This is how I end:

root@Ubuntu-1604-xenial-64-minimal-app:/var/www/discourse# rake posts:remap['^(#+)([^#\\s].*)$','\\1 \\2','regex'] --trace
** Invoke posts:remap (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute posts:remap
Are you sure you want to replace all regex occurrences of '^(#+)([^#\s].*)$' with '\1 \2'? (Y/n)
rake aborted!
NoMethodError: undefined method `acting_user=' for nil:NilClass
/var/www/discourse/lib/post_revisor.rb:132:in `revise!'
/var/www/discourse/app/models/post.rb:471:in `revise'
/var/www/discourse/lib/tasks/posts.rake:141:in `block in remap_posts'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:63:in `block (2 levels) in find_each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:63:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:63:in `block in find_each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:129:in `block in find_in_batches'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:230:in `block in in_batches'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:214:in `loop'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:214:in `in_batches'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:128:in `find_in_batches'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/relation/batches.rb:62:in `find_each'
/var/www/discourse/lib/tasks/posts.rake:133:in `remap_posts'
/var/www/discourse/lib/tasks/posts.rake:174:in `block in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/task.rb:251:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/task.rb:251:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/task.rb:251:in `execute'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/task.rb:181:in `invoke'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:153:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:109:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:109:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:109:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:118:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:103:in `top_level'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:81:in `block in run'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:179:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rake-12.1.0/lib/rake/application.rb:78:in `run'
bin/rake:4:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:75:in `load'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:75:in `kernel_load'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/cli.rb:424:in `exec'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/cli.rb:27:in `dispatch'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/cli.rb:18:in `start'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/exe/bundle:30:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/lib/bundler/friendly_errors.rb:122:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.16.1/exe/bundle:22:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => posts:remap

(Gerhard Schlager) #4

Hmm… looks like you found a post that doesn’t belong to a topic. :thinking:

(Markus) #5

How can I fix this? :thinking:

(Mittineague) #6

Will need to figure out how it came to be before it can be fixed.

If you have the Data Explorer plug-in installed that might help in figuring out what happened. You could also check out the Action Logs for the OP to see if it had been put into needs approval / Akismet queues, Flagged etc.

(Markus) #7

Isn’t it possible to skip this post without topic?

(Markus) #8

Merry X-mas dear friends,

(1) may I ask you: How do I use the data explorer? I‘ve never been worked with it before. Which queries can I use to identify the issue?

(2) Action log: Which events are suspicious?

(3) Akismet: Nothing in the queue.

(4) Console output: Why are you thinking there is a post without topic? I didn’t read anything like that? Why is the rake task saying the first method „acting_user“ is undefined?

Thanks in advance,

(Mittineague) #9

Merry Christmas right back at ya!

The way I use the Data Explorer plugin is I have a query named “test” that I use for “alpha” queries and when I find one I like rename it to something more descriptive and save it then create another “test”.

To the right of the input area there is a list of Discourse tables. They can be expanded to show field names with their data types.

I have limited experience with different databases, so my SQL isn’t so bad. But I still check the postgreSQL documentation often

IMHO the plugin does a great job of letting you know when something is wrong with the query syntax. And it has safety built in in case you run a resource hungry query. However, similar to poor regex, you need to get things right. As an analogy, if you tell it “I wants watere” it will give you “plural used incorrectly, no such thing as watere” messages, but if you tell it “I want water” when you want bread, it will happily give you water.

I’m in the habit of explicitly SELECTing the fields I want so I don’t know if the “*” wildcard will work, but it may. eg.

FROM posts 
WHERE topic_id IS NULL 

should let you know how many of these you have.

(Markus) #10

Thank you very much! It seems the Data Explorer is a very mighty tool.

Unfortunately, I‘ve got as output count = 0. Looks like, there is something different wrong. :thinking:

  • Regarding to this hint by @sam , I also did a rebuild of the app container. (no success)

(Gerhard Schlager) #11

It’s failing to set the topic’s acting_user because @topic is nil.

You could try the following Data Explorer query to find the problematic posts.

FROM posts
    SELECT 1
    FROM topics
    WHERE posts.topic_id =

Anyway, I changed the rake task to continue remapping when an error occurs.

(Markus) #12

Thank you very much, @gerhard! :rose:

When I try to open them, I‘m getting the message: The site ist not existing or private. And nearly everyone has <p>...</p> in common. That‘s intressting, cause I‘ve never put html code into these posts.

What is the best option:

  1. Can I recover / repair them? It seems the content is still available.
  2. If not, how can I delete them manually / automatically?

Some example:

Thank you in advance!

(Markus) #13

Yessssss :confetti_ball::christmas_tree: :heart: Now it‘s working @gerhard, you are my hero :blush:

(Stephen Chung) #14

Now you only have the one problem left… You have a post, or posts, that have incorrect topic_id field set to a number that doesn’t correspond to a valid topic, or a topic that is abnormal.

If the topic_id is invalid, then you have data inconsistencies in your database, which may trip up your other stuff in the future. You really need to get to the bottom of this.

If the topic_id points to a valid topic, then it is simply something that the rake task hasn’t handled, which is safe.

(Markus) #15

For now, I’ve no clue how to fix this issue.
It seems there is some data behind each topic but those entries can’t be restored or deleted.

Any suggestions how to solve this?

(Gerhard Schlager) #16

Your screenshot gives the impression that only “About the category” topics are affected. Did you delete them at some point in time? I just gave it a try and it looks like deleting those topics isn’t possible anymore. Maybe it was in the past and there was a bug…

If you really want those posts gone, you could delete them in the Rails console (rails c inside the Docker container). This should work:

Post.where("NOT EXISTS (SELECT 1 FROM topics WHERE posts.topic_id =").delete_all

No, it’s definitely not a problem with the rake task. The referenced topic IDs simply do not exist anymore.