Configure the Discourse RSS Polling Plugin

The Discourse RSS Polling plugin allows topics to be automatically created from one or more RSS feeds. This topic describes how to install and configure the plugin.

Installation

If your site is hosted by us, the plugin is pre-installed on your site, but the setting to enable it is hidden. Contact our support team to have us enable the plugin for you.

For self-hosted sites, install the plugin following the method outlined in Install Plugins in Discourse.

Configuration

For self-hosted sites, select the rss polling enabled site setting to enable the plugin. For sites on our hosting, contact out support team to have us enable the plugin for you.

The rss polling frequency site setting controls the frequency that the RSS feed is polled. The default value is to poll the feed once every 30 minutes. The setting has a minimum allowed value of 10 minutes.

Once the plugin is enabled, go to your Admin / Plugins page. Click the RSS Polling entry from the plugin’s page sidebar to add your feed URLs. Each Feed URL needs to be associated with a Discourse user. That user will be the author of the feed’s topics. Click the “save” icon to save your changes:

Set the feed’s categories

The categories that your feed gets published to are configured on the Host section of the Admin / Customize / Embedding page. Start by clicking the “Add Host” button to add a host record for your feed:

In the form that appears, set the “Allowed Hosts” field to the domain that is set in the link attributes of your feed. In many cases, the domain that is used in the link attributes will be the same domain as your feed is on. This is not always the case though. You can confirm this by loading your feed directly in your browser and looking at the value of the link attributes.

Ignore the host record’s “Class Name” field. That field is only used when Discourse comments are embedded in an external site with javascript.

The optional “Path Whitelist” field allows you to target specific URL paths to have them publish to separate Discourse categories. The path should start with / and end with a wildcard symbol ( .* ). For example /en/.* .

The “Post to Category” field sets the category that the host record will publish to.

In the following example, there are two host entries for the feed at blog.example.com . Posts in that feed that are on the /en/ path will be published to the “English” category. Posts in that feed that are on the /fr/ path will by published to the “French” category:

Showing the full post content

If the embed truncate site setting is enabled on your site (it is enabled by default), feed topics will only display an excerpt of the feed post. Users can view the full post by clicking the “Show Full Post” button that is displayed below the excerpt. Discourse will then scrape the topic’s embed URL to get the full post content. If the full post content is not being correctly retrieved, you can fix the issue by configuring the embed whitelist selector site setting. See How to configure the embed whitelist selector setting for details about that setting.

18 Likes

Is there a way to set the Discourse topic author to the post author? I seem to recall some way to pull the username from the RSS but I don’t see that setting anymore (discourse embed feed key or something like that?). We use SSO login and we’d like to set the topic author properly so that replies will notify the original author.

We’re trying this plugin out in an attempt to solve the embedding errors that we began encountering sometime after February 2020… more detail at Embedding error.

1 Like

I seem to remember that it used to be possible to pull the topic author from the feed. That functionality doesn’t exist in the RSS Polling plugin though. I’m not sure how much work it would be to have the plugin attempt to pull the creator from the feed and match them with an existing Discourse user.

2 Likes

It looks like the embed_username_key_from_feed SiteSetting was removed here: FEATURE: Replace SimpleRSS with Ruby RSS module (#5311) · discourse/discourse@5f318a5 · GitHub

I could try submitting a pull request to this plugin that adds it back as a plugin setting but might need a little hand holding :sweat_smile:

Looks like most of the work would be in feed_item.rb and poll_feed.rb as well as adding the setting to feed_setting.rb

2 Likes

Make sure you have enabled the rss polling enabled site setting. After doing that, try doing a hard refresh on your browser window. You should then see an “RSS Polling” entry in the sidebar on your Admin / Plugins page.

1 Like

I am not certain where you are having the trouble with this. When you add the feeds, your RSS Polling setting should look like this, but with a different username:

Make sure to click the Save icon. The icon used for the Save button is easy to miss. Clicking it doesn’t give the user any feedback that the settings have been saved. Possibly this is something that could be improved.

You’ll need to create two embeddable Host entries for the feed and then set the Path Allowlist to get the feed to publish to the correct category. It looks like the feed from cnn_tech.rss is served from this path: /~r/rss/cnn_tech/.* and the path for the feed at money_latest.rss is /~r/rss/money_latest/.*

Try configuring your embeddable hosts in a way similar to this to see if that gets the feed into the correct categories:

1 Like

Would it be possible to change this plugin to add Discourse tags to each RSS item in addition to category?

Thank you.

3 Likes

Is it possible to use any other variables in embed.imported_from apart from %{link}?

I’d like to create a link using title and URL separately (in the default message the URL is displayed in full as a URL).

Thanks.

When I delete topics created via RSS they always seem to rise from the dead fairly quickly. What should I do to prevent this? Thanks.

I’m going to reduce Customise/Embedding “Maximum number of posts to embed” still further in the hope that this helps.

Edit: I reduced that to 3 and deleted eight posts. All eight were simultaneously resurrected.

Maybe the answer is just not to delete a topic until it ceases to be available in the RSS feed.

==============================

31 Oct 2020: Can’t reply as I’m talking to myself here so editing the post :slight_smile:

I now don’t think this is relevant to the RSS plugin.

I’d still like an answer to this :slight_smile:

The answer was to unlist the topics so that they wouldn’t be hoovered up from the RSS feed again.

1 Like

The path that is used is set by the value of the link attribute that is set for each feed item. I think I found the paths for the feed at rss.cnn.com by adding http://rss.cnn.com/ to my site’s Feed URL setting and then looking at the embed URLs of the topics that were created by it.

1 Like

I’m still interested in customising this message. Thanks in advance for any assistance.

I’d like to use RSS Polling to auto-post videos from YouTube playlists, which are in the form: https://www.youtube.com/feeds/videos.xml?playlist_id=ID.

However, the link attributes are the videos themselves, like https://www.youtube.com/watch?v=VID. This means I can’t use these URLs to set the categories by regex, but I could with the source/RSS URLs (each RSS would target a category). This isn’t possible right now, right?

1 Like

It’s not currently possible. There are some details about the issue here: Discourse-rss-polling: polling YouTube playlist feed.

2 Likes

Ok, so just to be clear: it isn’t possible to use YouTube feeds because they use some custom media tags, right?

However, I’ve seen similar situations to what I described above on blog posts too, where the link from the feed isn’t enough to identify a category, so in these cases it’s not possible to auto-set a category?

Right. I see now that I didn’t answer your initial question:

You should be able to get all feeds that have their link attribute set to the www.youtube.com domain to publish to a category by creating a host record with its domain set to www.youtube.com: Something like this:

I don’t think there is a way to set specific categories based on the playlist’s ID though.

1 Like

Browsing through the forum today I wondered again about this. The reason I ask is that some URLs are long and messy and it would good to “hide” them behind some more useful text in an HTML a tag.

The feed url section does not appear. I tried hard refresh. what could be the problem?

Click the “Plugins” link from the admin navigation menu again. You should now see an “RSS Polling” entry in the left-hand sidebar:

It is on its own route, instead of being set via the normal site settings pages. I’m not sure if that’s something that could be changed. It would be more straightforward if the Feed URL could be set as a normal site setting.

2 Likes

Oh, it seems now. I am sorry for the confusion. Thank you.

1 Like