Configure the Discourse RSS Polling Plugin

[Jonathan5]

It’s quite greedy isn’t it? You just need to unlist (manually) instead of deleting.

[simon]

@CaptainZac

It’s posting 20 topics at a time, but it’s the same topics getting posted every time. I delete them, but then they will show back up in 10 minutes.

As @Jonathan5 noted, if you delete topics that still exist in the feed, Discourse will recreate the topics the next time it pulls in the feed. Unlisting the topics is probably the best way to deal with that.

If the exact same 20 posts keep getting pulled in, it seems likely that those are the only posts that exist in the feed.

[simon]

@renato

I’d like to use RSS Polling to auto-post videos from YouTube playlists, which are in the form: https://www.youtube.com/feeds/videos.xml?playlist_id=ID .

Apparently you can pull in a YouTube channel’s feed with the plugin. Possibly the same approach will work for a playlist. Have a look at this post for details: Is there a YouTube plugin - #3 by CaptainZac. If you get a chance to try this out, let us know if it works.

[daemon (Alex)]

@simon

The plugin works pretty well, but I have one problem.

All new “posts” are invisible until I set them manually to visible.

Or is this same as you here described?

@simon

You can prevent this by disabling the embed unlisted site setting. That setting is enabled by default. When enabled, topics that are embedded from external sources will be unlisted until they receive a reply.

[simon]

@daemon

Or is this same as you here described?

Yes, you need to disable the embed unlisted site setting so that new topics created from the RSS feed will not be set as unlisted. I am going to update the first post in this topic to add instructions for disabling that setting.

[daemon (Alex)]

@simon

Thank you!

But I need to say that the translation to german is not as good as it should be, because “unlisted” is not the same as “invisible” and that’s why I’m a little bit confused.

EDIT:
I had a look and in the context the translation is ok.
But in the topic itself you can set the topic visible/invisible and it does the same.
So why are here two different texts?
Or to ask a little bit OT: Is in Discourse listed/unlisted the same as visible/invisible?

[P16]

@simon and for anyone else having issues with getting images to work:

1. Use Zapier or something similar to strip HTML tags from the content, and then publish feed
2. Enable markdown for embedded posts in your site settings

Images will now work as usual.

[Whiskey_Victor_LLC]

I’m not understanding this part. I am getting the feed articles, but they are only viewable by the admin (they appear to be not listed) and they are not going into the correct categories.

And I don’t understand what this means - " Replace REPLACE_ME with the canonical URL of the page you are embedding it on."

[simon]

@Whiskey_Victor_LLC

I am getting the feed articles, but they are only viewable by the admin (they appear to be not listed) and they are not going into the correct categories.

To fix the issue with the topics being set as unlisted, disable your Discourse embed unlisted site setting.

The issue with the topics not being published to the category that you have set is probably happening because the feed’s posts are not being served from the feed’s domain. You will need to look at the raw feed, by loading the feed directly in your browser. Once you have loaded the feed, look for the value of the feed’s link attributes. The domain that is used in this attribute needs is what needs to be added to the Allowed Hosts record that you create. Have a look at Configure the Discourse RSS Polling Plugin for details.

@Whiskey_Victor_LLC

And I don’t understand what this means - " Replace REPLACE_ME with the canonical URL of the page you are embedding it on.

You can ignore that section of the Embedding page. That code is used for displaying Discourse comments on an external site.

[Whiskey_Victor_LLC]

Thanks! I found the unlisted setting and made that change, all go now.

So I deleted the feeds I had and readded a couple and one of them appears to be working and posting in the correct category. However, even though I have “Maximum number of posts to embed” set to 10, it downloads 1400 posts the first time I add it.

Also, can you explain how to use the Crawler Settings in a little more detail?

Thanks again for the assistance!

[Jonathan5]

It will download every single item in the RSS feed. If you delete a topic it’ll just recreate it. The only way to reduce the number of items it sucks up is to remove them from the RSS feed at source. Once it’s imported something, you can unlist the topic instead of deleting it.

[simon]

Yes, unfortunately the Maximum number of posts to embed setting is not applied to RSS feed items.

[Jonathan5]

@Jonathan5

@Jonathan5

Is it possible to use any other variables in embed.imported_from apart from %{link}?

Browsing through the forum today I wondered again about this. The reason I ask is that some URLs are long and messy and it would good to “hide” them behind some more useful text in an HTML a tag.

I’m still interested to hear about this Thanks.

[simon]

@Jonathan5

Is it possible to use any other variables in embed.imported_from apart from %{link}?

No, the value that is supplied to the %{link} placeholder is a URL that’s passed to the placeholder in the Discourse code. Changing the name of the placeholder will not have an effect on that.

[Jonathan5]

Thanks. Could it be converted into a link along the lines of DuckDuckGo instead of https://duck.com/?

[simon]

This won’t work with the current code. The method that’s creating the link is:

  def self.imported_from_html(url)
    "\n<hr>\n<small>#{I18n.t('embed.imported_from', link: "<a href='#{url}'>#{url}</a>")}</small>\n"
  end

That will always create a link with the link’s URL set as the link text.

[Jonathan5]

Thanks again. Is that the sort of thing that can be easily overridden by a plugin? I’d like to change the whole string! I’ve never written a plugin but maybe in the long term…

[Jonathan5]

Also, the topic date seems to be import date rather than RSS feed item date. Is there any way to change this? It’s not a problem until a change is made to an RSS item (eg description) at source, at which point it becomes the latest post, out of order.

Edit: the answer to this was to change the category’s default sort order to created descending! It hasn’t changed on the main Categories page but I assume it will catch up within 24 hours or so.

[simon]

@Jonathan5

Is that the sort of thing that can be easily overridden by a plugin?

It’s been a while since I wrote a plugin. I believe that it would be possible to re-open the method in a plugin and customize it’s HTML output.

@Jonathan5

Also, the topic date seems to be import date rather than RSS feed item date

That is correct. The feed item date is not used when creating the topic.