Discourse-indexnow — Instantly notify Bing & Yandex when topics are created or edited

Customization Plugin
1
:information_source: Summary Automatically submits new and edited public topic URLs to the IndexNow protocol, so Bing, Yandex, and other participating search engines can discover your content in minutes instead of waiting for a crawl.
:hammer_and_wrench: Repository Link github.com/imlotso/discourse-indexnow
:open_book: Install Guide How to install plugins in Discourse

A Chinese-language writeup with more background is also available on my forum: sitetalk.net/t/topic/642.


Features

  • Automatically submits new public topics, and re-submits when an edit changes the topic’s first post or key properties.
  • When Content Localization and its crawler locale parameter are enabled, localized topic URLs are automatically included alongside the main URL — using Discourse’s real locale query parameter (never hardcoded).
  • Submits the primary URL and all existing localizations together in a single IndexNow urlList batch request, rather than one request per URL.
  • Historical backfill: filter topics by category and date range, preview the matching results, then submit them in one batch — reusing the same batching engine, which automatically splits submissions over the protocol’s 10,000-URL limit.
  • Manual submission: paste one URL per line to submit specific links on demand; external or ineligible URLs are filtered out automatically.
  • Hourly/daily submission caps, with automatic backoff when IndexNow returns a 429 (respects Retry-After when present).
  • Key rotation support — the previous key stays valid for a 7-day transition window so in-flight submissions don’t break.
  • Built-in reachability check for your /<key>.txt file, shown directly in the admin panel.
  • Automatically re-submits (or excludes) topics when they’re moved between categories, or when a category’s visibility/tags change.
  • Submission logs track batch ID, locale, a 7-day success/failure trend, and a breakdown of failure reasons (rate limit / key error / domain mismatch / other).
  • All submissions run asynchronously via Sidekiq and go through the same privacy filters as automatic submissions — nothing bypasses the exclusion rules (private messages, restricted categories, unlisted/deleted topics, login-required sites).
  • Admin UI available in English and Simplified Chinese.

Configuration

  1. Install the plugin (see the Install Guide above) and rebuild your container.
  2. Go to Admin > Plugins > discourse-indexnow.
  3. On the Settings tab, click Generate key (or paste an existing 32-character hex key you’ve already registered elsewhere).
  4. Check the Enable IndexNow submissions box.
  5. Confirm the key is publicly reachable at:
    https://your-forum.example/<key>.txt
    It should return HTTP 200 with the key as plain text. The admin panel also runs this check automatically and shows a cached green/red status next to “key.txt accessible”.
  6. (Optional) On the Logs tab, use the Backfill panel to submit historical topics — pick a category and date range, click Preview to see how many topics match, then Submit.
  7. (Optional) Use the manual submission field to paste one-off URLs, one per line, for immediate submission outside the normal automatic flow.

The Logs tab also shows a 7-day success/failure trend chart and a breakdown of failure reasons, so you can tell at a glance whether submissions are being rate-limited, rejected for a key mismatch, or failing for another reason.

Settings

Name Description
indexnow_enabled Master switch for the plugin. Cannot be enabled while login_required is on, and auto-disables if login_required is turned on afterward.
indexnow_api_key Current 32-character hex key used to authenticate submissions.
indexnow_submit_on_create Automatically submit when a new public topic is created.
indexnow_submit_on_edit Automatically re-submit when the first post or topic attributes change.
indexnow_excluded_category_ids Additional categories to always exclude, even if they’re public.
indexnow_hourly_limit Maximum number of URLs submitted per hour.
indexnow_daily_limit Maximum number of URLs submitted per day.

Known limitations

  • IndexNow has no delete notification — topic deletions are only logged for audit purposes, not submitted.
  • Google does not participate in the IndexNow protocol.
  • Localized URLs are only submitted when the crawler locale parameter is enabled and a localization actually exists for that topic.

This is an independent, unofficial plugin — not maintained by the Discourse team. It’s running in production on my own forum, has RSpec test coverage with CI on every push/PR, and I’d appreciate any bug reports or feature requests via GitHub issues or replies here.

1 Like