# Load test with Discourse
**URL:** https://meta.discourse.org/t/load-test-with-discourse/77291
**Category:** Development
**Created:** [January 5, 2018, 7:31am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291 "2018-01-05T07:31:57Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [January 5, 2018, 7:31am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/1 "2018-01-05T07:31:57Z")
Hi,
I am doing a load test for Discourse.
Here is my scenario:
1. Take [Discourse Meta](https://meta.discourse.org/latest) for example. I append a api\_key (generated from admin portal) as a query string. So the url I test is: [Discourse Meta](https://meta.discourse.org/latest?api_key=XXXXX);
2. I remove the api rate limit by disable rate.limit.template in app.yml and set DISCOURSE\_MAX\_ADMIN\_API\_REQS\_PER\_KEY\_PER\_MINUTE=10000;
3. Run the url with multiple agents.
Here is the result:
And the CPU of postgreSQL reach 100%.
This is the result without api\_key as query string:
As far as I know, the anonymous request will get cached result from Redis, while the authenticated request will not. But the load test with authenticated request is lower than my expectation.
Is there any wrong in my steps of load test? Or setting I missed?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [January 5, 2018, 8:29am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/2 "2018-01-05T08:29:43Z")
Can you expand a bit on the test setup, how many unicorns, how is pg setup and so on…
---
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [January 5, 2018, 8:55am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/3 "2018-01-05T08:55:02Z")
1. Front end: Azure Web Service for Container(Standard S3, 4 core, 7GB RAM) 10 instances, each have 8 unicorn workers.
2. Cache: Azure Redis Cache Standard
3. SQL: Azure PostgresQL service (preview) Standard, 800 Compute Units. It’s about a 8 core VM.
To give more information, I found that the result vary from different api\_keys.
1. api\_key A is the admin user, which is used to create 1.5 million topics load test data.
2. api\_key B is a normal user, which has zero usage.
The api\_key A runs load test very slow as the original post. While the api\_key B run much faster.
Here is the throughput from api\_key B:
I suspect there is a slow query in user related operation which cause the issue.
---
### Author:  [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [January 5, 2018, 9:28am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/4 "2018-01-05T09:28:33Z")
If the PostgreSQL CPU is reaching 100%, then there’s your bottleneck. You’ll presumably need a bigger database instance, or dig into what exactly the database is doing that’s so slow (via slow query logs, or otherwise).
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 5, 2018, 2:24pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/5 "2018-01-05T14:24:29Z")
Also, you may need to tune PostgreSQL to use your memory instead of the disk for sort operations and such.
The settings `work_mem` and `shared_buffers` are the principal ones.
---
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [January 5, 2018, 2:39pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/6 "2018-01-05T14:39:23Z")
Yes, the slow query is the a join operation of table `topic` and `topic_user_ids`. When user has created a lot of topics before, the operation of show latest page will be slow. I will try to tune it by using better PostgresSQL and `work_mem` and `shared_buffers`. Thanks @mpalmer nd @Falco
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 5, 2018, 2:43pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/7 "2018-01-05T14:43:48Z")
I dunno how your load test works but in a real use, when a lot of users hit your site, the content is already there, and PostgreSQL had time to pass the `VACUUM` process everywhere, so stats are up to date and the planner plans are optimal. If you are creating thousands of topics/posts and reading at the same time, you may need more aggressive (or even manual) `VACUUM` triggers.
Also, if the disks on the Azure PostgreSQL are **fast** you may need to lower `random_page_cost` to a similar value of `seq_page_cost`.
Disclaimer: I may have a PostgreSQL T-shirt that I use to sleep sometimes.
---
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [January 5, 2018, 3:39pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/8 "2018-01-05T15:39:25Z")
I am not doing read/write at the same time. The load test reads from 1.5 million topics while no write operation occurs.
Does `fast` means SSD? Then yes, it’s SSD 😄
---
### Author:  [@mcwumbly](https://meta.discourse.org/u/mcwumbly)
#### Post date: [January 5, 2018, 3:53pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/9 "2018-01-05T15:53:08Z")
> [@SimonWu](#):
>
> When user has created a lot of topics before, the operation of show latest page will be slow.
You may want to consider having your load test use a pool of users that create posts, which might better simulate real world traffic and database state.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [January 5, 2018, 11:33pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/10 "2018-01-05T23:33:18Z")
The usual traffic breakdown is very high anon traffic percentage. I just checked boingboing and ~~for the last 30 days, 400k logged in pageviews vs 4.3m, anon so about 10%.~~ whoops that was wrong, not sure where I was looking there 😖
Last 30 days all traffic, **4.7 million** pageviews, of which **351,000** were logged in users. So 7.5% of traffic is logged in users.
---
### Author:  [@notriddle](https://meta.discourse.org/u/notriddle)
#### Post date: [January 6, 2018, 12:21am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/12 "2018-01-06T00:21:45Z")
> [@Falco](#):
>
> Disclaimer: I may have a PostgreSQL T-shirt that I use to sleep sometimes.
Is this supposed to be read as “I have an unhealthy obsession with PostgreSQL”, or is it supposed to mean “The sum total of my PostgreSQL knowledge is a shirt that I got at a con”?
---
### Author:  [@mpalmer](https://meta.discourse.org/u/mpalmer)
#### Post date: [January 6, 2018, 12:29am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/13 "2018-01-06T00:29:05Z")
---
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [January 9, 2018, 2:17am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/14 "2018-01-09T02:17:43Z")
Quite similar, for our scenario, we have 10% traffic logged in users.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 9, 2018, 2:22am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/15 "2018-01-09T02:22:55Z")
Hey @SimonWu,
Our tests are showing around 3x performance gains when running PostgreSQL 10 in some slow queries, thanks to the new parallel scans.
Do you know how close Azure is on the PostgreSQL 10 upgrade?
Some examples:
```plaintext
Query 1 PG 9.5: 1258.871 ms
Query 1 PG 10: 293.879 ms
Query 2 PG 9.5: 15532.489 ms
Query 2 PG 10: 4889.282 ms
```
---
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [January 9, 2018, 3:06am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/16 "2018-01-09T03:06:09Z")
The Azure PostgreSQL is not GA yet. I don’t know if upgrade to 10 in in their roadmap
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 8, 2018, 4:36am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/17 "2018-03-08T04:36:26Z")
Curious, where is Postgres at on Azure these days? What is the most powerful instance you can get hardware wise?
---
### Author:  [@SimonWu](https://meta.discourse.org/u/SimonWu)
#### Post date: [March 9, 2018, 5:50am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/18 "2018-03-09T05:50:11Z")
For now, it’s server with 32 cores. There will be 64 cores machine in recent months.
---
### Author:  [@ryanerwin](https://meta.discourse.org/u/ryanerwin)
#### Post date: [April 12, 2018, 1:22pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/19 "2018-04-12T13:22:35Z")
Note, for anyone trying to do their own Discourse load testing like Simon, not only do you need to edit the MAX\_ADMIN\_API\_REQS, you also need to comment out `web.ratelimited.template.yml`
Something like:
#### /var/discourse/containers/app.yml
```plaintext
templates:
- "templates/postgres.template.yml"
- "templates/redis.template.yml"
- "templates/web.template.yml"
# - "templates/web.ratelimited.template.yml"
...
env:
LANG: en_US.UTF-8
DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE: 10000
...
```
Otherwise, you’ll probably hit the nginx rate limits on requests per IP address. If you see response code `429` (Too many requests) even after increasing the MAX ADMIN API REQS… this is why.
Also note that YAML variables use “:”, so if you copy and paste Simon’s `DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE=10000` you’ll get a YAML parsing error about a colon…
---
### Author:  [@pradeepconnecpath](https://meta.discourse.org/u/pradeepconnecpath)
#### Post date: [August 13, 2018, 5:44pm UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/20 "2018-08-13T17:44:01Z")
Hi Ryan, Do I need to rebuild the discourse for this change to become active?
---
### Author:  [@ryanerwin](https://meta.discourse.org/u/ryanerwin)
#### Post date: [August 15, 2018, 10:15am UTC](https://meta.discourse.org/t/load-test-with-discourse/77291/21 "2018-08-15T10:15:44Z")
Yes, you must do a rebuild of the container to activate this change.
Note that applies to any `app.yml` changes since they’re only read once, while the container is `built`.