# Plugin to scrape news from other sites
**URL:** https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973
**Category:** Support
**Created:** [February 12, 2022, 1:16pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973 "2022-02-12T13:16:36Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@f1r4s](https://meta.discourse.org/u/f1r4s)
#### Post date: [February 12, 2022, 1:16pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/1 "2022-02-12T13:16:36Z")
Hello,
Is there any plugin to scrab news from other sites ?
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [February 12, 2022, 1:58pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/2 "2022-02-12T13:58:55Z")
There’s: [News Plugin 📰](https://meta.discourse.org/t/news-plugin/107971)
Which uses RSS feeds
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [February 12, 2022, 2:16pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/3 "2022-02-12T14:16:05Z")
The [Configure the Discourse RSS Polling Plugin](https://meta.discourse.org/t/configure-the-discourse-rss-polling-plugin/156387) can scrape many sites, like blogs, youtube channels and playlists, and automatically create new topics just a few minutes after content is posted.
---
### Author:  [@f1r4s](https://meta.discourse.org/u/f1r4s)
#### Post date: [February 13, 2022, 12:51pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/4 "2022-02-13T12:51:18Z")
I installed the plugin; and follow the guide as below;
i put 3 sections to test working rss from bleeping security site;
> 1st category rss source
> [News in the Security category](https://www.bleepingcomputer.com/news/security)
> category filter: none.
> username selected; and category to post rss selected.
> second category rss source
> [BleepingComputer](https://www.bleepingcomputer.com/feed/)
> category filter: security
> username selected; and category to post rss selected.
also i use another site rss
> [darkreading](https://www.darkreading.com/rss.xml)
> category filter: vulnerabilities-threats
> username selected; and category to post rss selected.
but nothing work with me ??
---
### Author:  [@tobiaseigen](https://meta.discourse.org/u/tobiaseigen)
#### Post date: [February 13, 2022, 1:26pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/5 "2022-02-13T13:26:36Z")
Did you go to sidekiq to run the process? If not then it will take a while, depending on your admin settings. Also look in the logs for any errors.
You also have to be sure to use an rss url. If you open thise urls they don’t seem to be rss. There is also a setting that makes imported topics unlisted by default.
---
### Author:  [@f1r4s](https://meta.discourse.org/u/f1r4s)
#### Post date: [February 13, 2022, 1:43pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/6 "2022-02-13T13:43:08Z")
sorry, i miss that part; the logs show me this
---
### Author:  [@f1r4s](https://meta.discourse.org/u/f1r4s)
#### Post date: [February 13, 2022, 1:49pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/7 "2022-02-13T13:49:08Z")
I see now only the third rss ([darkreading.com](http://darkreading.com)) is working to start posting; and it’s post pretty good more than 100+ post, but all the posts looks like this
value:
```plaintext
https://www.darkreading.com/rss.xml
category filter: vulnerabilities-threats
username selected; and category to post rss selected.
```
how can i include full post with pictures ?
---
### Author:  [@justin](https://meta.discourse.org/u/justin)
#### Post date: [February 13, 2022, 4:57pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/16 "2022-02-13T16:57:39Z")
@f1r4s and @Jagster, keep it civil here.
While there are legitimate uses for wanting to pull in content from other sites, such as for an internal community where you want to monitor important security news, we do not condone copyright infringement.
Discourse community owners are responsible to run their site in accordance with all governing laws and host terms of service, just like with any other site on the web.
---
### Author:  [@justin](https://meta.discourse.org/u/justin)
#### Post date: [February 13, 2022, 4:59pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/18 "2022-02-13T16:59:13Z")
Temporarily closing this out for a cool down.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [February 13, 2022, 5:04pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/19 "2022-02-13T17:04:49Z")
> [@f1r4s](#):
>
> how can i include full post with pictures ?
Try toggling the `embed truncate` site setting.
> [@f1r4s](#):
>
> also i use another site rss
>
> …
>
> but nothing work with me ??
Some sites have a weird RSS markup, so you will have to manually debug broken ones.
---
### Author:  [@justin](https://meta.discourse.org/u/justin)
#### Post date: [February 14, 2022, 2:00pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/20 "2022-02-14T14:00:18Z")
This topic was automatically opened after 21 hours.
---
### Author:  [@f1r4s](https://meta.discourse.org/u/f1r4s)
#### Post date: [February 15, 2022, 4:11pm UTC](https://meta.discourse.org/t/plugin-to-scrape-news-from-other-sites/217973/21 "2022-02-15T16:11:56Z")
I recommended using second [RSS but the images are broken](https://meta.discourse.org/t/configure-the-discourse-rss-polling-plugin/156387/113) in scrapping..