# freeCodeCamp.org Discourse is Collapsing from Spammer Scripts

**URL:** <https://meta.discourse.org/t/freecodecamp-org-discourse-is-collapsing-from-spammer-scripts/155738>\
**Category:** Self-hosting\
**Created:** [June 23, 2020, 9:53pm UTC](https://meta.discourse.org/t/freecodecamp-org-discourse-is-collapsing-from-spammer-scripts/155738 "2020-06-23T21:53:17Z")\
**Posts on this page:** 1\
**Showing post:** 1

<div class="post-metadata">

**Author:** ![ossia](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ossia/32/215651_2.png) [@ossia](https://meta.discourse.org/u/ossia)\
**Post date:** [June 23, 2020, 9:53pm UTC](https://meta.discourse.org/t/freecodecamp-org-discourse-is-collapsing-from-spammer-scripts/155738/1 "2020-06-23T21:53:17Z")

</div>

We started seeing heightened loads on the [freeCodeCamp.org](http://freeCodeCamp.org) Discourse on June 21. Our average CPU load started to grow to the point that the entire forum became unresponsive.

## Our Initial investigation

1. We updated Discourse from the Stable version to the latest Tests-Passed (Beta) version, which is recommended to get all the latest performance fixes. There seemed to not be much of a difference.

2. We got rid of some of the old plugins to help diagnose the issue, and it does not seem to be the case for slow down.

3. To rule out any misconfigurations, we tested this with the default theme and checked the `/logs` and did find some exceptions:

This seems to be because the instance may already stressed out with a lot of long running jobs.

The discourse container (upstream) is running on Digital Ocean, and seems to be running hot on all 6 vCPUs. It is currently running on version: 2.5.0.beta7

# Further investigation

1. This morning our moderators let us know that there was a significant increase in the number of spam accounts, which was leading us to believe that this could be a targeted attack.

2. To try and mitigate the issue, we have put the forum into read-only mode. Despite this, the resource utilization rate remains quite high at 60+%.

3. From what we can tell: ever since this started, we have been seeing a lower requests-per-second and higher current request rate on our proxy:

 ![](https://global.discourse-cdn.com/meta/original/3X/3/a/3a56fbb94a033db710fc117f12de1868a34f21e8.png)

1. The volume of traffic to [The freeCodeCamp Forum - Join the developer community and learn to code for free.](http://freecodecamp.org/forum) has not significantly changed according to Google Analytics.
2. None of the other applications on the same proxy seem to be affected. Our /news and /learn platform are operating normally.
3. We tried adding some additional rate limiting on the proxy, and it did not make much difference, so we removed it (to allow normal users to continue to reach our site).
4. We have also moved our old [forum.freecodecamp.com](http://forum.freecodecamp.com) redirects away from the .org forum since we noticed that old subdomain (which we haven’t used in 2.5 years) had a big spike in traffic starting a few days ago.

At the moment with ~400 real-time users, the stats look like so:

 ![](https://global.discourse-cdn.com/meta/original/3X/c/f/cf6eedfd9096aab833f5493679ef325ef2e56e4c.png)

## Observations About The Spammer’s Behavior

The spammers seem to be running some sort of script that creates new accounts on a Discourse instance, then waits several days. Then those accounts suddenly become active. They start posting new threads with links to a website. (Perhaps to build backlinks?)

Some of these accounts were created all the way back in March.

Here’s what one of their spam posts look like, though there are a lot of variants, linking to a lot of websites:

 ![signal-2020-06-23-164925_003](https://global.discourse-cdn.com/meta/original/3X/9/d/9dd12042431ea742782f52457e962f48f922c738.png)

Any advice would be welcome here.

---

_[View the full topic](https://meta.discourse.org/t/freecodecamp-org-discourse-is-collapsing-from-spammer-scripts/155738)._
