Issues with embedding from RSS

I’m having a few issues with embedding from RSS. I’m going to enable comments for each original post, but I want to ensure that the comments end up in the right location in Discourse with the correct attributed author and publishing date.

Issue 1: embed username key from feed

I need each topic to be correctly attributed to the right author. I’m set up as the default. My RSS has a format of <discourseUsername>jo</discourseUsername>, so I entered discourseUsername as the value for embed username key from feed. It just attributes all topics to the default user. Have I done something wrong here?

Issue 2: pubdates

I need each topic’s timestamp to be the same as the original post publishing time. Otherwise adding our RSS feed of ~70 posts floods the forum with outdated topics that have recent timestamps. (Pushing newer manually-created topics down.) Is there a way to set the timestamp for topics pulled from RSS to the same time as the RSS <pubdate>?

Issue 3: chronology

Maybe related to Issue 3, the topics imported via RSS are not in the same chronological order as the original posts.

I hope none of these are stupid questions! Any help, suggestions, or workarounds would be much appreciated! :smile:

3 Likes

We will take a look, @eviltrout is working on improved embedding for this release.

3 Likes

Thanks! It’d also be pretty cool if you could feed different (multiple) RSS feeds into different categories, or put RSS-fed topics into different categories dependent on a key value.

Our use case is that we have a couple of blogs, one “newsy” and one “developer-y”, I’m using a combined RSS feed for now so that we can pull them all into Discourse, but having their separate posts/topics in their own categories would make it easier for audiences who favour one blog or the other.

2 Likes

That looks correct to me. Can you give me a sample of the RSS feed you’re trying to import?

Another option is to use the embed by username site setting if you are okay with all posts coming from the same user.

This is unfortunately not possible right now. The embedding tool was originally meant to be “going forward” so it doesn’t have support for dates via embedding. It’s something we could definitely look at adding. (This is the same as issue 3 as far as I can tell).

Feed is https://ind.ie/rss with items structured like:

<item> <title>The Social Web: A Glorious Dystopia</title> <link>https://ind.ie/blog/the-social-web-a-glorious-dystopia/</link> <guid>https://ind.ie/blog/the-social-web-a-glorious-dystopia/</guid> <description>&lt;p&gt;This week we&apos;re going to dive straight into the activities of those cheeky corporations. There&apos;s so much to share!&lt;/p&gt; &lt;p&gt;&lt;a href=&quot;https://ind.ie/blog/the-social-web-a-glorious-dystopia/&quot;&gt;Read the full post on the Ind.ie blog&lt;/a&gt;.&lt;/p&gt;</description> <pubDate>Fri, 24 Jul 2015 18:30:00 GMT</pubDate> <author>laura@ind.ie (Laura Kalbag)</author> <discourseUsername>laura</discourseUsername> </item>

I’ve got the embed by username set to my username, but also got embed username key from feed. Will these conflict? I’d rather have them all assigned to the original author, not just me. Though I write the most blog posts, so having it set to me isn’t a bad default…

That makes sense. We’re going to use it “going forward” now too :smile:

It’s not exactly the same as issue 3 (but would have the same solution.) Even though the RSS is imported all at once, the topics are still ordered (because that’s how they’re presented in Discourse.) However it’s unclear what the topics are ordered by, as it’s not the same order as the <item>s in the RSS.

Hi!

I also have issues with embedding RSS.

These are my settings:

Yet all my RSS posts ends up in the category “Uncategorized”. Have I done something wrong here?

Steps to repdocude:

  1. Create an embedded feed (in this case from svedala.se)
  2. Set the real RSS address in the settings (not just the …/nyheter/)
  3. Post as discobot
  4. Set 5 minutes polling interval
  5. Wait 10(!) minutes for it ti poll
  6. See that posts are posted in “Uncategorized”
2 Likes

Same problem here.
I’ve tried a number of times to get this plugin working.
All the topics are created in the uncategorized category.

I’ve turned the plugin off and it is still working!
I’m sick of deleting topics.
They might be the same old feeds as I’m beginning to recognise the titles I’ve delete x5.
Do I have to edit the app.yml and rebuild to stop the plugin?

I’d rather it worked as expected as it would be very useful.
Has anyone got this running with categories?

Sorry for the late reply to this. The category that RSS feed items get posted to is based off the domain that is used in the feed item’s link attribute. It is not based off the domain of the RSS feed. For example, if your RSS feed is at www.example.com, but the link attribute in your RSS feed is pointing to blog.example.com, you would set the category for the feed by adding a host entry for blog.example.com.

If people are running into issues with RSS categories that can’t be solved with the above approach, let us know and we’ll look into what is going on.

I will update the copy that is used by the plugin to make it clear how to set categories for the feed’s posts.

This needs to be fixed. The plugin should not continue to create topics from the RSS feed when it is disabled. For now, you should be able to stop topics from being created by deleting the host entry that you have set for the plugin’s domain.

@simon

Sorry if I was unclear. The full URL is: Nyheter

1 Like

When I look at your RSS feed, the link attributes for the feed items are using the www.svedala.se domain. The host record in your screenshot is for the domain svedala.se. Try adding a host record for www.svedala.se and setting its category to RSS. Can you let us know if it still doesn’t post to the correct category after making this change?

One thing I need to confirm is whether or not you need to create a host record for the RSS feed’s domain, or if host records only need to be added for the domain’s that are used in the feed’s link attributes. I will test this out on my own site and report back here.

1 Like

OK, so this is the current setting:

Does that look more accurate to you?

The domain that you have set in the Allowed Hosts section looks correct. I think the Path Whitelist setting you have is wrong though. The purpose of that setting is to allow you to publish posts that are at different paths on your website to specific Discourse categories. You can leave the setting blank if you want to publish all posts from www.svedala.se to your RSS category.

If you want to have different paths from the domain publish to different Discourse categories, the path that you add should end in with a wildcard symbol. For example, /paverka/nyheter/.*

2 Likes

Thank you for your attention on this, I will let it run for some time and get back to you.:+1:

1 Like

Hi I’ve done that. I’m still getting feeds in the uncategorised category.
RSS Polling Plugin off
Embedding completely empty and asking for me to add a host.

Oh no. We will update the plugin so that disabling the plugin completely stops topics from being created from the feed.

1 Like

I’ve pushed a fix for this issue to the plugin’s repository. That code will get reviewed and hopefully added to the plugin soon.

For now, the advice I gave you about disabling the feed was wrong. To prevent feed topics from being created, you need to re-enable the plugin, then go to the RSS Polling settings link. On that page, delete the Feed URL that you have entered.

Once you have done that, feed topics should no longer be created.

3 Likes

Hi Simon thanks.

I’m a little unclear.
Have you fixed the feeds going into the uncategorised category or the feeds not stopping?

I fixed the issue with the feeds not stopping when the plugin is disabled. I also updated the text that the plugin displays when you set a Feed URL to make it clear how to set the category for a feed’s topics. The category that a feed gets published to is based off the domain of the feed’s link attributes. That is not something that can be changed.

The PR I made has not been merged yet. It should be available soon though.

2 Likes