Hi, I am doing a load test for Discourse. Here is my scenario: Take Discourse Meta for example. I append a api_key (generated from admin portal) as a query string. So the url I test is: Discourse Meta ; I remove the api rate limit by disable rate.limit.template in app.yml and set DISCOURSE_MAX_AD…

Can you expand a bit on the test setup, how many unicorns, how is pg setup and so on…

Front end: Azure Web Service for Container(Standard S3, 4 core, 7GB RAM) 10 instances, each have 8 unicorn workers. Cache: Azure Redis Cache Standard [image] SQL: Azure PostgresQL service (preview) Standard, 800 Compute Units. It’s about a 8 core VM. To give more information, I found that the res…

If the PostgreSQL CPU is reaching 100%, then there’s your bottleneck. You’ll presumably need a bigger database instance, or dig into what exactly the database is doing that’s so slow (via slow query logs, or otherwise).

Also, you may need to tune PostgreSQL to use your memory instead of the disk for sort operations and such. The settings work_mem and shared_buffers are the principal ones.

Yes, the slow query is the a join operation of table topic and topic_user_ids. When user has created a lot of topics before, the operation of show latest page will be slow. I will try to tune it by using better PostgresSQL and work_mem and shared_buffers. Thanks @mpalmer nd @Falco

I dunno how your load test works but in a real use, when a lot of users hit your site, the content is already there, and PostgreSQL had time to pass the VACUUM process everywhere, so stats are up to date and the planner plans are optimal. If you are creating thousands of topics/posts and reading at …

I am not doing read/write at the same time. The load test reads from 1.5 million topics while no write operation occurs. Does fast means SSD? Then yes, it’s SSD :smile:

[image] SimonWu: When user has created a lot of topics before, the operation of show latest page will be slow. You may want to consider having your load test use a pool of users that create posts, which might better simulate real world traffic and database state.

The usual traffic breakdown is very high anon traffic percentage. I just checked boingboing and for the last 30 days, 400k logged in pageviews vs 4.3m, anon so about 10%. whoops that was wrong, not sure where I was looking there :confounded: Last 30 days all traffic, 4.7 million pageviews, of which…

[image] Falco: Disclaimer: I may have a PostgreSQL T-shirt that I use to sleep sometimes. Is this supposed to be read as “I have an unhealthy obsession with PostgreSQL”, or is it supposed to mean “The sum total of my PostgreSQL knowledge is a shirt that I got at a con”?

Load test with Discourse

Development

SimonWu (Simon Wu) January 9, 2018, 2:17am 14

Quite similar, for our scenario, we have 10% traffic logged in users.

Topic		Replies	Views
Long loading times for user summary page with slow database Self-hosting unsupported-install	17	2147	October 29, 2019
Slow Profile Loads with 100GB+ database Feature	42	3590	December 29, 2020
PostgreSQL runaway IO Self-hosting	35	3711	March 3, 2021
Slow performance due to per query latency connecting to database? Self-hosting	8	987	September 11, 2018
Slow SQL query causes homepage to load in 2-4 sec Product Support	15	1836	February 7, 2018

Load test with Discourse

Related topics