Thanks for building this. The feature set is genuinely useful and I want to keep… using it. I ran it on a live Discourse forum and hit a set of problems, some of them severe. I verified each of these against the code rather than reporting them from symptoms alone. Everything below refers to `v0.2.0`.
I have fixes for most of these and I am happy to send them as PRs if you want.
## 1. The admin route path collides with other plugins and can take the browser down
**Severity: High. The admin panel becomes unusable and the tab can run out of memory.**
`assets/javascripts/discourse/admin-indexnow-route-map.js`:
```js
export default {
resource: "admin.adminPlugins.show",
map() {
this.route("discourse-indexnow", { path: "logs" });
},
};
```
`adminPlugins.show` is mounted at `/plugins/:plugin_id`. Using `path: "logs"` does not restrict it to IndexNow's logs page. It claims `/admin/plugins/<ANY_PLUGIN>/logs` across the entire site. Any other installed plugin that uses `path: "logs"` will collide with it.
On my forum `discourse-sitemap-autolink` did exactly that. Straight from the browser console:
```js
Discourse.__container__.lookup("service:router")
.recognize("/admin/plugins/discourse-indexnow/logs").name
// => 'adminPlugins.show.discourse-sitemap-autolink-logs'
```
Visiting IndexNow's logs page rendered the *other plugin's* page under the IndexNow heading.
**Why this escalates to a browser crash:** Core's `AdminPluginsShowIndexRoute.afterModel` (`frontend/discourse/admin/routes/admin-plugins/show/index.js`) does this:
```js
afterModel(model) {
if (this.adminPluginNavManager.currentPluginDefaultRoute) {
this.router.replaceWith(
this.adminPluginNavManager.currentPluginDefaultRoute,
model.id
);
}
}
```
There is no check that the route exists. If the advertised nav route cannot be resolved, this throws on *every* visit to `/admin/plugins/discourse-indexnow`. Ember retries the route and the DOM grows on each retry. What I saw was:
```text
Error: There is no route named adminPlugins.show.discourse-indexnow
at CO.afterModel (index.js:14:19)
```
This was followed by the page rendering the site header over and over, scrolling forever, until the browser tab ran out of memory. Deactivating the plugin immediately made the admin panel accessible again.
**Suggested fix:** Give both the route name and the path a plugin-specific prefix, like `this.route("discourse-indexnow-logs", { path: "indexnow-logs" })`. Naming the child route after the plugin id is also worth avoiding since it reads like the plugin's index route.
*(Two upstream Discourse issues are arguably in play here too: `mapRoutes()` gives no warning when two plugins claim the same child path, and `replaceWith` above has no existence check. The plugin can avoid both by scoping its path.)*
## 2. The admin nav tab is registered without checking if the route resolves
`assets/javascripts/initializers/indexnow-admin-plugin-configuration-nav.js` calls `api.addAdminPluginConfigurationNav(...)` unconditionally.
Because of the unguarded `replaceWith` mentioned above, advertising a nav entry whose route failed to register is not a cosmetic problem. It is the difference between a missing tab and a completely broken admin panel. Confirming the route resolves before advertising it (`router.recognize(url)`) turns a hard crash into a gracefully absent tab.
## 3. Rate limits can be exceeded by 2x due to clock-based resets
**Severity: High. This is the issue most likely to get a site rate-limited by IndexNow.**
`lib/discourse_index_now/throttle.rb` keys its counters by clock hour and date:
```ruby
def hourly_key(time = Time.zone.now)
"indexnow:rate:hourly:#{time.strftime('%Y%m%d%H')}"
end
```
The counter drops to zero on the clock boundary rather than as the counts age out. The full hourly cap can go out at 1:59 and the full cap again at 2:01. Run against the current code:
```text
spend 200 at 01:59 -> at 02:01 used=0 capacity=200
```
That pushes 400 URLs in two minutes against a configured cap of 200/hour. The same trick works across midnight for the daily cap, letting `indexnow_daily_limit` run twice over in a few minutes.
IndexNow answers sustained over-submission with `429 Too Many Requests (potential Spam)`.
**Suggested fix:** Sum fine-grained buckets covering a trailing window instead of using one bucket per calendar period. 60 one-minute buckets for the hourly cap and 24 one-hour buckets for the daily cap keeps the read to a single `MGET` of a fixed 84 keys. Cost does not grow with submission volume. Numbering buckets off the epoch rather than a formatted clock string also removes the time-zone dependency in the current keys.
## 4. The admin stylesheet is served globally and contains unprefixed selectors
`plugin.rb`:
```ruby
register_asset "stylesheets/admin.scss"
```
Without the `:admin` scope, this is linked on every forum page for every visitor, including anonymous ones, for a stylesheet only the admin panel uses.
That would just be wasted bandwidth, but `assets/stylesheets/admin.scss` contains:
```scss
.indexnow-controls,
.status-filters,
.url-search,
```
`.status-filters` and `.url-search` are generic enough to match theme or core markup. Shipped site-wide, they can restyle unrelated elements on public pages.
**Suggested fix:** Use `register_asset "stylesheets/admin.scss", :admin` and prefix those two selectors (`.indexnow-status-filters`, `.indexnow-url-search`).
## 5. The JSON API returns an HTML page to non-JSON requests
`config/routes.rb` declares the management endpoints without a format default:
```ruby
scope "/admin/plugins/discourse-indexnow",
module: "discourse_index_now",
constraints: ::StaffConstraint.new do
get "/logs.json" => "admin_logs#index"
...
```
These actions inherit `Admin::AdminController`, and `check_xhr` runs before the controller's own filters. A request that is neither an XHR nor explicitly asking for JSON gets the admin SPA HTML shell with HTTP 200.
Confirmed by test with the format default removed:
```text
GET /admin/plugins/discourse-indexnow/logs.json (Accept: */*)
expected: "application/json"
got: "text/html"
```
Scripts, `curl`, and uptime checks get a full HTML page where they asked for data. Because `check_xhr` fires first, `requires_plugin` never runs, meaning a disabled plugin still answers 200.
**Suggested fix:** Add `defaults: { format: :json }` to that scope.
## 6. Category and tag changes fan out synchronously inside the web request
`lib/discourse_index_now/submission_service.rb`:
```ruby
def self.handle_category_updated(category)
...
topics = Topic.where(category_id: category.id)
if category.read_restricted?
topics.find_each { |topic| mark_topic_logs_failed(topic, "category_restricted") }
else
topics.find_each do |topic|
enqueue_topic(topic, localized: false, trigger_reason: :category_changed)
end
end
end
```
This runs in the request that saved the category. Each `enqueue_topic` writes submission log rows and enqueues its own Sidekiq job. Flipping the visibility of a category with N topics costs N jobs and N sets of DB writes before the admin's save returns. On a large category, that is a guaranteed request timeout.
`handle_tag_updated` has the same shape and lacks a `saved_change_to_*` guard. Any tag update, including editing a tag's description, resubmits every topic carrying that tag.
**Suggested fix:** Move both into a background job and batch the URLs rather than spawning one job per topic.
## 7. Generating a key destroys a working key with no confirmation
`generateKey` POSTs straight to `/generate_key.json`:
```js
async generateKey() {
try {
this.loading = true;
await ajax("/admin/plugins/discourse-indexnow/generate_key.json", { type: "POST" });
```
One click on a live install replaces a key that is currently working, with no prompt and no undo.
This goes beyond UX. After a rotation, IndexNow answers `202 URL received. IndexNow key validation pending` until it refetches `/<key>.txt`, and drops those URLs if validation fails. An accidental click quietly costs you submissions.
**Suggested fix:** Add a confirmation dialog when a key already exists. Discourse's `dialog` service has `deleteConfirm` for exactly this. Skip the prompt when the field is empty.
## 8. HTTP 202 is logged as "Successful" identically to 200
`lib/discourse_index_now/client.rb`:
```ruby
expects: [200, 202],
...
{ success: true, status: response.status }
```
Per the IndexNow spec these mean different things:
* **200**: URL submitted successfully
* **202**: URL received. IndexNow key validation pending
202 is not an acceptance. If key validation fails, those URLs are discarded. I hit this directly on my forum. Submissions came back 202, never showed up in Bing, and the admin log reported them as Successful alongside genuine 200s.
The `response_code` is already stored on the log row, making this a display fix rather than a schema change. As it stands, an admin cannot tell landed submissions from pending ones.
## 9. Setting a limit to 0 blocks submissions and busy-loops the job queue
`available_capacity` returns 0 when either limit is `<= 0`:
```ruby
return 0 if hourly_limit <= 0 || daily_limit <= 0
```
Here `0` means "submit nothing", not "unlimited". This is a reasonable choice, but neither setting has a `min:` constraint in `settings.yml` and the admin UI does not clarify this. An admin who sets 0 expecting no limits gets total silence with no explanation.
Worse, `SubmitBatch` reschedules itself when capacity is 0, and `next_window_delay` has no branch for this case:
```ruby
if hourly_limit.positive? && hourly_remaining <= 0 # false when limit is 0
...
retry_delay || 60
```
The job reenqueues itself every 60 seconds indefinitely. Because it uses a fresh `Jobs.enqueue_in` rather than a Sidekiq retry, the `retry: 5` limit does not apply.
**Suggested fix:** Enforce `min: 1` on both settings, or add an explicit branch that fails the batch with a clear reason instead of looping.
## Not a bug (noting so nobody chases it)
**Replies are never submitted.** `handle_post_created` returns early on anything that isn't the first post:
```ruby
return unless post.is_first_post?
```
I initially read this as an oversight, but I was wrong. Crawlers get the paginated view (`app/views/topics/show.html.erb`, chunked at `TopicView::CHUNK_SIZE`). Page 1's content genuinely only changes when the first post, title, category, or tags change. Submitting the topic's base URL on every reply asks search engines to refetch an unchanged page. That is exactly the pattern that earns a 429.
The real gap is narrower. Later crawler pages do change on a reply, and those are never submitted. On a long topic, `/t/slug/id?page=17` is its own canonical URL and search engines never hear that it changed. Submitting `?page=N` for the page the reply landed on rather than the base URL addresses that without the over-submission the current gate protects against.
I have forked the repo and will be submitting fixees that address these issues and add a few more features and options.