How to rebake only certain posts?


#1

When rebaking an imported forum (eg. one with >300k posts) is there a way to flag the posts:rebake or refresh_oneboxes with specific post_ids that we want to rebake? When using rake posts:refresh_oneboxes, we can get to around 200,000 before it seems memory problems causes problems with the task. The posts already in the queue continue to complete successfully. So really we just want to issue the command for post_ids higher than 200,000 rather than run through all of them again.

We have 8GB of RAM currently on the server, and our app.yml looks like this:

env:
  LANG: en_US.UTF-8
  ## How many concurrent web requests are supported?
  ## With 2GB we recommend 3-4 workers, with 1GB only 2
  UNICORN_WORKERS: 4
  UNICORN_SIDEKIQS: 3
  ##
  DISCOURSE_DB_POOL: 25

(Régis Hanol) #2

It’s unfortunately not possible with the current version of that rake task.

We would happily accept a PR adding that option though :wink:


#3

Ah, okay. Is there any way to increase the amount of memory available to the Sidekiq processes? I think that is why it is crashing.


#4

Just an update to this. I have found that one can easily do this by editing the posts.rake file in the discourse/lib/tasks folder.

You need to find the line:

  Post.find_each do |post|

And edit this to:

Post.where(id: <first_post_id>..<last_post_id>).find_each do |post|

For example, if you want to rebake all posts with an id of between 20000 and 20100 inclusive you would put:

Post.where(id: 20000..20100).find_each do |post|

Then run the rebake command as per normal (rake posts:rebake or rake posts:refresh_oneboxes). That will just bake those 100 posts.

Of course, back up the original posts.rake file and restore it once you are finished. Using this method I was able to finish off rebaking the forum after an import so all the images, links etc. are looking pretty!


(Jeff Atwood) #5

It might be nice if the admin wrench on a post offered a “rebake” option, eventually.


(Régis Hanol) #6

The rebake option is now available :monkey:

https://github.com/discourse/discourse/commit/e56fcf0c43828088e4b50fea674d5474ab823239


(Arpit Jalan) #7