Feed polling: How it works?


#1

Hi guys! quick question.

I’ve configured a rss feed and also enabled feed polling. Reloaded sidekiq, and begin to wait for news on my forum, but nothing happends. Even I trigered manually the event (using Sidekiq’s panel) and nothing happings.

Am I missing something?

TkS!


(Jeff Atwood) #2

What URL did you specify for the rss endpoint?


#3

I’ve specified an RSS feed directly. Is that correct? thanks for the feedback :smile:


(Nick Gray) #4

I can’t seem to get this working either. Here is the RSS feed I am trying to use:

http://store.steampowered.com/feeds/news.rss

The category to post under is set to ‘Sweet Deals - Free Stuff’. Not sure if this requires a category ID from the database, a URL or what.


(Jeff Atwood) #5

I literally have no idea what you guys are talking about. There is no support built into Discourse for creating topics from RSS feeds, if that is what you are trying to do.


(Nick Gray) #6

What does the RSS feed polling feature do then? Maybe I am just mistaken at its actual function but the verbage in the settings makes it sound like it takes RSS content and post it in a category.


#7

I’m confused also with this functionallity. I thought that setting an RSS Feed on this settings, Sidekiq will pull the feed and post the new message in the board. Is that correct?


(Neil Lalonde) #8

Which setting are you enabling exactly?


#9

This response gave me a chuckle…I think they want to look into the API stuff if they want to pull from discourse to whatever website that is not discourse (blog or personal site).

I could be wrong though…I’ve been wrong once…and maybe one other time.


(Nick Gray) #10

Here is the settings page under embedding which makes it look even more like you are polling RSS feeds and embedding it into your site:

This even says “URL of RSS/ATOM feed to import”. How does this not look like what we are talking about?

Further, looking at the feed polling function in discourse it appears to do exactly what we think it does:


(Jeff Atwood) #11

I will change the copy here to be more clear.


#12

Thanks for quoting that piece of code :smile:

My install was not working because I did not setup email_by_username field. Once I configured it, it started to work.


(Nick Gray) #13

Interesting. I do not see email_by_username in settings there. Care to point me in the right direction?

Edit: Seems I have it working as well but the RSS content was not complete. It parsed the information a little strangely. I went ahead and tweaked the CSS element setting to see if that helps.

To add: I had to set a valid existing user to have the posts show up. I had to create a user for the RSS feed import and post as it.


#14

Apologies. It is “embed by username”. Try looking for “embed” in the search textbox.


(Nick Gray) #15

Ah indeed. Found the option, just wanted to make sure that was a typo and I wasn’t missing anything.

Were you able to import a full RSS feed with content? I think it may be the way the steam RSS feeds are, but all I get is a link to the steam store application page. I was wondering if there are any tweaks to get actual content to import rather than just a 1 line post.


(Michael Downey) #16

So I’m lost here. :confused: Are these features about importing RSS feed items as new topics, or are these settings about something else?


(Nick Gray) #17

I was able to get those settings to import RSS news feed items after setting correct settings. I wasn’t however, able to get content in the post but I suspect it is due to crappy xml on the steam news rss feed.

Unsure of what @codinghorror was talking about, the RSS feed import works exactly how we thought it would.


(Nick Gray) #18

Sorry to resurrect this post, but in light of the steam summer sales, I tried to start polling again.

Here is what I was trying to explain in the last few posts about the RSS content being ignored and it simply pulls a link to the RSS news item.

While this is at least a start, the actual RSS content looks as follows:
http://store.steampowered.com/news/13700/

<item rdf:about="http://store.steampowered.com/news/13700/">
<title>
<![CDATA[ Now Available on Steam - Beyond Space ]]>
</title>
<link>http://store.steampowered.com/news/13700/</link>
<guid isPermaLink="true">http://store.steampowered.com/news/13700/</guid>
<comments>http://store.steampowered.com//forums/</comments>
<pubDate>Wed, 18 Jun 2014 01:28:24 -0700</pubDate>
<author>Valve</author>
<category>Valve news update</category>
<dc:subject>Valve news update</dc:subject>
<dc:creator>Valve</dc:creator>
<dc:date>2014-06-18T01:28:24-07:00</dc:date>
<content:encoded>
<![CDATA[
<a href='http://store.steampowered.com/app/297100/'>Beyond Space</a> is Now Available on Steam!<br><br>Our journey begins when Max Walker starts a contract with the Alliance of Planets and immediately gets drawn into a galaxy-spanning conflict. <br /><br><br /><br>His mission will take him beyond the reaches of space, while fighting many different opponents and meeting pirate overlords, scheming politicians, deceitful allies and ambitious leaders &#150; all entangled in a space epic that will change the face of the galaxy.<br><a href="http://store.steampowered.com/app/297100/"><img src="http://cdn.akamai.steamstatic.com/steam/apps/297100/capsule_467x181.jpg" style=" float: left; margin-right: 12px; height: 181px; width: 467px;"></a>
]]>
</content:encoded>
</item>

It also pulls posts in a strange order. Seems to take any random post it wants and imports them in whatever order it pleases.

I am also a little confused at what this setting will do (sorry for the black on grey the admin panel requires a hackjob to get CSS elements to display correctly and I just haven’t gone through all the little snippits of elements yet) :

There isn’t much instruction on what this feature is used for. I am assuming it will allow me to embed the rest of the news item but I have no idea how it will allow me or what format it needs to be. It seems the normal trend for options is to use pipe delimited lists in settings, but that seems to not have much effect.


(Jeff Atwood) #19

I hope this copy change helps clarify what this is for:

embeddable_host: "Host that can embed the comments from this Discourse forum"
feed_polling_enabled: "EMBEDDING ONLY: Whether to embed a RSS/ATOM feed as posts"
feed_polling_url: "EMBEDDING ONLY: URL of RSS/ATOM feed to embed"
embed_by_username: "Discourse username of the user who creates the embedded topics"
embed_username_key_from_feed: "Key to pull discourse username from feed"
embed_truncate: "Truncate the embedded posts"
embed_category: "Category of embedded topics"
embed_post_limit: "Maximum number of posts to embed"
embed_whitelist_selector: "css selector for elements that are allowed in embeds"
embed_blacklist_selector: "css selector for elements that are removed from embeds"

This feature is not meant as a general way to pull in RSS feeds as topics, it’s meant for embedding Discourse comments on another site.


Simple-rss is buggy and should be forked or replaced
(Nick Gray) #20

I see. That explains why there is only little bits of the RSS feed it is polling. Guess this is a job for a plugin! Perhaps I’ll brush off my ruby-fu and see if I can muster something up.