I’m continuing this conversation from ‘ Rebuild HTML for entire topic ’ as my experiments are going into quite another direction and I thought there might be value in sharing my thoughts and results as I go along. My situation is the following: we’re on the brink of launching a new migrated forum wit…

We should normalize the rake task to go in descending ID order as well @techapj . Unless this is super hard, many hours of work, or something?

Agree, but it is a bit tricky cause we would need to carry a big list of ids in memory. I wonder if we should amend it so the rake task is resumable? Have rake posts:rebake reset version and just work through old posts using calls to rebake_old And add rake posts:rebake:resume that simply resumes …

[图片] sam: a big list of ids in memory Is carrying a list of integer IDs in memory really that expensive?

[图片] codinghorror: Is carrying a list of integer IDs in memory really that expensive? we can probably live with it to be honest … that retains the tasks working exactly as they do today (in reverse order). Though something in me wants these tasks to be resumable cause if you are working throu…

Maybe V1 can be the simple version with a comment // TODO: make this resumable because carrying around 20 million ids in memory is not a great idea long term

Done via: FIX: rake task should rebake posts in descending ID order committed 06:48PM - 09 Apr 18 UTC (UTC) [图片] arpitjalan +6 -3 …

I’ve used a script that was resumable at the topic level by using the custom fields. Here’s one that skips private messages (since my import had a LOT of them and they weren’t a priority): Topic.includes(:_custom_fields).where(archetype: Archetype.default).find_each do |t| unless t.custom_fields[…

[图片] pfaffman: So rather than running rake posts:rebake, one should instead do Posts.all.update_all(‘baked_version: null’) and all posts will be rebaked in batches according to rebake_old_posts_count? Yes, and @bartv would be able to get his “rebuild for just one topic” by doing: Posts.where…

What’s the frequency of these new batches, and how can you monitor the progress?

[图片] sam: but it is a bit tricky cause we would need to carry a big list of ids in memory. I wonder if we should amend it so the rake task is resumable? This is now done via: PERF: do not carry post ids in memory when rebaking all posts …

我在一个大型帖子重建工作中的旅程

支持安装

techAPJ (Arpit Jalan) 2018 年4 月 9 日 18:53 17

Done via:

4 个赞

话题		回复	浏览量
Rake posts:rebake_incremental (feature request) Feature	2	793	2020 年4 月 22 日
Rebake all posts? Self-hosting	16	9044	2022 年10 月 28 日
Slow rebake script to avoid overwhelming your server Migration	4	142	2025 年11 月 13 日
Rebake with rails command or rake task doesn't work, but rebuilding HTML does. Why? Self-hosting	11	1918	2022 年4 月 26 日
Rebuild HTML for entire topic Feature	6	3893	2018 年4 月 8 日

我在一个大型帖子重建工作中的旅程

相关话题