I’m continuing this conversation from ‘ Rebuild HTML for entire topic ’ as my experiments are going into quite another direction and I thought there might be value in sharing my thoughts and results as I go along. My situation is the following: we’re on the brink of launching a new migrated forum wit…

We should normalize the rake task to go in descending ID order as well @techapj . Unless this is super hard, many hours of work, or something?

Agree, but it is a bit tricky cause we would need to carry a big list of ids in memory. I wonder if we should amend it so the rake task is resumable? Have rake posts:rebake reset version and just work through old posts using calls to rebake_old And add rake posts:rebake:resume that simply resumes …

[изображение] sam: a big list of ids in memory Is carrying a list of integer IDs in memory really that expensive?

[изображение] codinghorror: Is carrying a list of integer IDs in memory really that expensive? we can probably live with it to be honest … that retains the tasks working exactly as they do today (in reverse order). Though something in me wants these tasks to be resumable cause if you are work…

Maybe V1 can be the simple version with a comment // TODO: make this resumable because carrying around 20 million ids in memory is not a great idea long term

Done via: FIX: rake task should rebake posts in descending ID order committed 06:48PM - 09 Apr 18 UTC (UTC) [изображение] arpitjalan +6 -3 …

I’ve used a script that was resumable at the topic level by using the custom fields. Here’s one that skips private messages (since my import had a LOT of them and they weren’t a priority): Topic.includes(:_custom_fields).where(archetype: Archetype.default).find_each do |t| unless t.custom_fields[…

[изображение] pfaffman: So rather than running rake posts:rebake, one should instead do Posts.all.update_all(‘baked_version: null’) and all posts will be rebaked in batches according to rebake_old_posts_count? Yes, and @bartv would be able to get his “rebuild for just one topic” by doing: Po…

What’s the frequency of these new batches, and how can you monitor the progress?

[изображение] sam: but it is a bit tricky cause we would need to carry a big list of ids in memory. I wonder if we should amend it so the rake task is resumable? This is now done via: PERF: do not carry post ids in memory when rebaking all posts …

My journey into a massive posts rebake job

Поддержка Самостоятельное хостинг

techAPJ (Arpit Jalan) 09.Апрель.2018 18:53:05 17

Done via:

4 лайка

Тема		Ответов	Просм.
Rake posts:rebake_incremental (feature request) Feature	2	793	22.04.2020
Rebake all posts? Self-hosting	16	9044	28.10.2022
Slow rebake script to avoid overwhelming your server Migration	4	142	13.11.2025
Rebake with rails command or rake task doesn't work, but rebuilding HTML does. Why? Self-hosting	11	1918	26.04.2022
Rebuild HTML for entire topic Feature	6	3893	08.04.2018

My journey into a massive posts rebake job

Связанные темы