Why doesn't Discourse support IndexNow?

Feature
1

In today’s fast-paced and rapidly obsolete information, indexing speed is one of the most important factors for success.

But why doesn’t Discource support this protocol at all? https://www.indexnow.org/

2 Likes
2

Because no one cared enough to build a plugin or a pull-request to support it. Which I’d say it’s probably caused by the fact that Google doesn’t support IndexNow, which is the search engine most people care about.

But if you want to build a plugin to add this feature that’s a welcome contribution!

10 Likes
3

I would like to contribute to the community and program this extension, but we are not programmers.

Google’s attitude towards IndexNow is that they are testing it and we’ll see.

4

Any news on Index Now? Now that even Open AI run a search engine that draws from the Bing index and that is linked to IndexNow, it makes even more sense.

5

Then you could commission the plugin in Marketplace. I can imagine solutions in the $500 to $2000 range. Others might have better imaginations than I.

3 Likes
6

Or…

You could try with ChatGPT :sweat_smile:… I gave it a try:

Creating a plugin for Discourse to support the IndexNow protocol involves the following steps. IndexNow allows websites to notify search engines about updates (additions, deletions, or changes) to their content. Below is a detailed guide and the code for a basic plugin.
Steps to Create the Plugin

Set up the Plugin Folder:
    Create a folder in your Discourse installation under plugins. For example: plugins/discourse-indexnow.

Create the Plugin Files:
    Add the necessary Ruby and configuration files.

Add Functionality:
    Capture content updates in Discourse (like new posts or topic updates).
    Notify IndexNow endpoints using HTTP POST requests.

Test the Plugin:
    Ensure it works in your local environment and follows the IndexNow requirements.

Plugin Code Example

Here is a simple implementation for the discourse-indexnow plugin:

  1. plugin.rb

Create the main plugin file to register the plugin and add hooks.

# frozen_string_literal: true

# name: discourse-indexnow
# about: Notify IndexNow when Discourse content changes
# version: 0.1
# authors: Your Name
# url: https://github.com/yourusername/discourse-indexnow

enabled_site_setting :indexnow_enabled

after_initialize do
  require 'net/http'
  require 'uri'
  require_dependency 'topic'
  require_dependency 'post'

  class ::IndexNowNotifier
    INDEXNOW_ENDPOINT = "https://www.bing.com/indexnow"

    def self.notify(urls)
      return unless SiteSetting.indexnow_enabled

      uri = URI(INDEXNOW_ENDPOINT)
      key = SiteSetting.indexnow_key
      body = { host: Discourse.base_url, key: key, keyLocation: SiteSetting.indexnow_key_url, urlList: urls }

      Net::HTTP.post(uri, body.to_json, "Content-Type" => "application/json")
    end
  end

  # Hook into topic creation
  Topic.register_topic_posted_callback do |topic|
    IndexNowNotifier.notify(["#{Discourse.base_url}/t/#{topic.slug}/#{topic.id}"])
  end

  # Hook into post creation
  Post.register_post_created_callback do |post|
    IndexNowNotifier.notify(["#{Discourse.base_url}/p/#{post.id}"]) if post.post_number != 1
  end
end
  1. config/settings.yml

Add settings for the plugin.

plugins:
  indexnow_enabled:
    default: false
    client: bool
  indexnow_key:
    default: ""
    client: text
  indexnow_key_url:
    default: ""
    client: text
  1. README.md

Document how to install and use the plugin.

# Discourse IndexNow Plugin

This plugin enables Discourse to notify search engines using the IndexNow protocol.

## Installation

1. Clone the repository into your Discourse `plugins` folder:

git clone https://github.com/yourusername/discourse-indexnow.git

2. Rebuild your Discourse container:

./launcher rebuild app

3. Enable the plugin in the Discourse settings.

## Configuration

- **IndexNow Key**: Add your IndexNow key.
- **IndexNow Key URL**: Add the URL where your key is hosted.

Notes

The plugin uses Discourse hooks to capture content changes.
Ensure you host your IndexNow key in a publicly accessible URL as per the protocol's requirements.
Test the plugin thoroughly in a staging environment before deploying it to production.

Would you like additional functionality or customizations?

7

have you tested it? I think it’s a pretty bad idea to post solutions directly from ChatGPT without first packaging and testing the solution … (hint: I can already see problems with this within seconds of looking at it)

8 Likes
8

Ok… easy, my friend, I’m no programmer and haven’t tested it, just thought it was something curious/interesting to post.

9

It’s not a terrible start (though a better start is to start with the plugin skeleton with a rake task that I can never remember). It’s missing the piece that validates the key, which is a, key piece.

3 Likes
10

I don’t think it’s a good idea to post raw ChatGPT output for complex scenarios like this, and here’s why:

Zero-shot ChatGPT output for coding is notoriously unreliable and usually full of mistakes, especially in long form.

In the era of AI, Posts may be consumed by RAG or worse training and potentially be regurgitated as gospel.

Far better to refine offline, test and post the results.

Then when AI passes through it has something to improve its performance, not the other way around.

4 Likes