Estou continuando esta conversa de ‘Rebuild HTML for entire topic’, pois meus experimentos estão tomando um rumo bastante diferente e achei que poderia haver valor em compartilhar meus pensamentos e resultados conforme avanço.
Minha situação é a seguinte: estamos à beira de lançar um novo fórum migrado com mais de 4 milhões de posts. Estes precisarão de um rebake quando fizermos a troca para o domínio final, e os posts também precisarão de processamento para garantir que as imagens estejam incorporadas corretamente, etc.
Minhas preocupações são:
O rebake não é um processo rápido. Ajustei nosso servidor de 16GB/6 núcleos, mas não consigo obter nada mais rápido do que 2-3 posts/segundo, o que significa que o rebake completo levará bem mais de 20 dias.
O rebake começa pelos posts mais antigos; eu preferiria começar pelos mais recentes para oferecer a melhor experiência possível à nossa comunidade (assumindo que os posts mais novos receberão mais tráfego).
Não há como ‘retomar’ o processo de onde parou, e tenho motivos para suspeitar que precisarei reconstruir pelo menos uma vez nos próximos 20 dias.
As tarefas de rebake vão para a fila padrão do Sidekiq e estou preocupado que isso crie grandes atrasos para as tarefas de processamento regulares.
Até agora, fiz o seguinte: depois de investigar o código e receber ajuda da equipe aqui, hackeei o arquivo lib/tasks/posts.rake para:
Trabalhar em ordem cronológica reversa, começando pelos posts mais recentes.
Exibir o ID atual do post/tópico para que eu possa facilmente adicionar à cláusula where da minha consulta e retomar o processamento em outro número de post.
Aqui está meu código:
def rebake_posts(opts = {})
puts "NEW Rebaking post markdown para '#{RailsMultisite::ConnectionManagement.current_db}'"
disable_edit_notifications = SiteSetting.disable_edit_notifications
SiteSetting.disable_edit_notifications = true
total = Post.count
rebaked = 0
ordered_post_ids = Post.joins(:topic)
.select('posts.id')
.where('topics.archetype' => Archetype.default)
.order("posts.id DESC")
.pluck(:id)
ordered_post_ids.in_groups_of(1000).each do |post_ids|
posts = Post.order(created_at: :desc).where(id:post_ids)
posts.each do |post|
rebake_post(post, opts)
print_status(rebaked += 1, total)
puts " > rebaking post id #{post.id} for topic id #{post.topic_id}"
end
end
SiteSetting.disable_edit_notifications = disable_edit_notifications
puts "", "#{rebaked} posts concluídos!", "-" * 50
end
Próximo passo: estou tentando descobrir como criar essas tarefas na fila de baixa prioridade. Dicas seriam muito bem-vindas
Now I’ve started my first large test, I noticed that the jobs processing has made several huge ‘steps’ in speed. I suspect this may have to do with a large number of my attached images having been moved to the tombstone - this is another ongoing project.
Yes I learned that too With the help of your team I figured out how to work around it though. I’m not sure this is a smart or even fast way of doing it, but it works for me.
Next issue: our new site will already go live while the posts:rebake job is running. Will having a large number of jobs in the default queue slow down regular site processes, and should I try to have posts:rebake start its jobs in the low priority queue instead? Or is this automatically handled?
So far, it seems that the queue that a job will be created in is a property of the job’s class, I’m not sure I could influence this in some way from within the posts.rake script?
If not, I’ll throttle the creation of new jobs to make sure the queue isn’t filling up.
I think there’s also a ‘version’ column on the posts table that you can null out to cause gradual rebaking, too. I think it does 100 posts every time the job triggers.
So rather than running rake posts:rebake, one should instead do Posts.all.update_all('baked_version: null') and all posts will be rebaked in batches according to rebake_old_posts_count?
Agree, but it is a bit tricky cause we would need to carry a big list of ids in memory. I wonder if we should amend it so the rake task is resumable?
Have rake posts:rebake reset version and just work through old posts using calls to rebake_old
And add rake posts:rebake:resume that simply resumes an interrupted rebake.
Downside here is that posts:rebake would unconditionally cause posts to rebake at some point in time even if the task is interrupted, but this may not matter.
we can probably live with it to be honest … that retains the tasks working exactly as they do today (in reverse order). Though something in me wants these tasks to be resumable cause if you are working through 20 million posts this can take many hours and if it breaks half way through it can be very frustrating to start from scratch.
I’ve used a script that was resumable at the topic level by using the custom fields. Here’s one that skips private messages (since my import had a LOT of them and they weren’t a priority):
Topic.includes(:_custom_fields).where(archetype: Archetype.default).find_each do |t|
unless t.custom_fields["import_rebake"].present?
t.posts.select(:id).find_each do |post|
Jobs.enqueue(:process_post, {post_id: post.id, bypass_bump: true, cook: true})
end
t.custom_fields["import_rebake"] = Time.zone.now
t.save
end
end
(This filled up Sidekiq’s default queue, so it’s not useful if you want to launch your site before the rebakes are completed.)
After they’re all done, all the TopicCustomField records with name “import_rebake” can be deleted.
With the help of your team I figured out how to work around it though. I’m not sure this is a smart or even fast way of doing it, but it works for me.