RSSポーリングでフィードの処理に20分以上かかる場合があります

zogstrip · 2026 年 9 月 15 日午前 8:27

大型フィードのポーリング/インポート速度を大幅に向上させるはずのPRがいくつか作業中です。

github.com/discourse/discourse

PERF: Use an index to look up embeds by URL in `TopicEmbed` (#43643)

main ← perf-topic-embed-url-lookup

merged 08:40AM - 15 Sep 26 UTC

ZogStriP

+58 -4

Previously, `TopicEmbed.topic_embed_by_url` matched `embed_url` with a case-inse…nsitive regex, which can't use an index, so every lookup scanned the whole `topic_embeds` table. RSS polling runs this lookup for every item of every feed on every poll, so re-polling an unchanged 1,600-item feed took ~230s on a site with 100k embeds. This change matches `lower(embed_url)` against the `http` and `https` variants of the normalized URL and adds an index on that expression, bringing the same re-poll down to ~5s. The oldest match is picked in Ruby instead of with `ORDER BY id LIMIT 1` because, until the new index has statistics, Postgres would walk the primary key rather than use it. Re-imports also compare `user_id`s instead of loading the post's user for every item. Reported at https://meta.discourse.org/t/412322

github.com/discourse/discourse

PERF: Use the embed URL index to find imported topics in RSS polling (#43644)

perf-topic-embed-url-lookup ← perf-rss-polling-imported-topics

merged 08:40AM - 15 Sep 26 UTC

ZogStriP

+22 -13

Previously, `ImportedTopics` matched every feed item URL in a single `embed_url …~* ANY(ARRAY[...])` query. Postgres only caches 32 compiled regexes, so with more patterns it recompiles them for every row it checks. With `rss_polling_update_tags` disabled, a 1,600-item feed took ~25 minutes per poll on a site with 100k embeds. This change looks topics up with `TopicEmbed.with_embed_urls`, which uses the `lower(embed_url)` index (~0.1s for the same feed), only loads the columns it needs, and builds lookup keys with `TopicEmbed.embed_url_key` so they can't drift from the core query. Reported at https://meta.discourse.org/t/412322 Stacked on https://github.com/discourse/discourse/pull/43643

トピック		返信	表示
Are there any upper limits to the RSS Polling plugin? Product Support rss-polling	6	351	2026 年 9 月 13 日
Rss polling frequency Product Support rss-polling	3	195	2025 年 6 月 12 日
A new admin experience for RSS Polling Announcements rss-polling	5	388	2026 年 7 月 24 日
RSS Polling Plugin official , rss-polling , included-in-core	4	18518	2026 年 8 月 14 日
RSS feed not read Product Support rss-polling	0	108	2025 年 6 月 13 日

RSSポーリングでフィードの処理に20分以上かかる場合があります

関連トピック