My discourse speed is very slow

ahmed_khan1 · March 3, 2021, 9:23am

our website was so slow we migrated it to a server our website with 4 core CPUs, 8GB Ram, and 8GB of swap memory our website still so slow. i can’t find a way to optimize my site

Don · March 3, 2021, 12:14pm

Less advert is good start I think.

pfaffman · March 3, 2021, 12:28pm

Wow

Wow. You weren’t kidding! It’s completely unusable. I want wasn’t even able to tell what the site was before I gave up.

Single cpu speed and SSD speed are big factors. How fast is the CPU? You don’t have spinning metal disks do you? Have you run discourse setup or changed the memory settings since your changed the amount of ram? Have you tweaked the database settings? You are using a Discourse official Standard Installation?

But since your site is mostly serving ads, it’s impossible to tell what the discourse performance is like.

ahmed_khan1 · March 3, 2021, 1:36pm

we are using ssds in our server

I have tested the site with ad blocker and speed is still slow

discourse is installed by digitalocean

no I haven’t

no!

pfaffman · March 3, 2021, 2:18pm

You should either run discourse-setup again or read the comments in app.yml to adjust the memory configuration.

You shouldn’t need any swap.

Is your database very big?

ahmed_khan1 · March 3, 2021, 4:06pm

i already have run discourse-setup that didn’t make any difference in speed

yes we have more than 27000 topics on our site

lhkjacky · March 3, 2021, 4:30pm

In my experience, Topic List Previews (Plugin) slow down discourse significantly. You may try switch to Topic List Previews (TC), it load much faster than Plugin.

If you disable the Topic List Previews completely, Discourse will speed up several times.

Faizan_Zahid · March 3, 2021, 5:54pm

We are using a custom plugin to allow topics to still show the cdn images shared on topics , in meta OG if we share them on social media. That cdn support plugin is developed by @fzngagan , faizan, do you have any way to move your written plugin to TLP TC so that site can load faster?

@pfaffman Do you have any specific tech suggestions whatexactly do we need to do to speed up our database and website, I m assuming on howtodiscuss.com it has over 50K+ topics and it’s a high traffic website.

My questions :

Is there any multi threading or parallel CPU or cores configuration we can set up in discourse ? nodebb provides this to cope up with high traffic.
Any server side caching support in discourse , where and how to configure it?
How can we achieve 100% Google page speedscore for https://howtodiscuss.com
what changes can we do in app.yaml file? we r on 4 Vcpu, 8GB digital ocean droplet.
we r using cloudflare to serve our site. any optimizations we can do there? when I turn on rocket loader of cloudflare, my discourse doesnt work properly and gives errors in js console client side.
any way to auto minify css, js, html, and enable gzip compression on server side?
Please share insights on what optimization you guys have done on meta.discourse.org to cope up with higher traffic, we can try the same.
Please share insights on optimizations which other high traffic sites on discourse have tried?

Once we are able to speed up our website, we are happy for you to post our site also in successful discourse communities list. Thanks

pfaffman · March 3, 2021, 5:59pm

Well, the number of topics isn’t the number of posts or the size of the database, but that isn’t so big.

The site is down now, so I can’t see the number of posts.

I recommend that you edit app.yml by hand and look at the tuning information there and rebuild. There are several topics that discuss ways to tune the database.

You might also switch to one of the new Digital Ocean faster CPU droplets.

dadberg · March 3, 2021, 6:23pm

Hey, @ahmed_khan1 Don’t risk any downtime, even few minutes down will cost you precious reputation in Google eyes.

Just pay someone professional to handle this for you, once and forever. As I can see your site got a sizeable traffic, so follow the sanity and never go for cheap.

I recommend someone like @pfaffman and @angus.

merefield · March 3, 2021, 9:40pm

I’m running a 20,000 Topic site on a 2 Core, 2GB machine with no issues (but may not be as active as yours …)

But yeah, if it’s an issue with running a lot of javascript, then the back end performance is irrelevant.

Have you set this to zero?

That will help a little.

You can move to the TC to speed TLP up, but it does have a rendering limitation in Chromium based browsers (hence why I’ve not decommissioned the plugin so far, Im waiting for LayoutNG to upgrade Grid CSS).

ahmed_khan1 · March 4, 2021, 8:16am

we have a new digitalocean droplet. just purchase it on 1
here’s app.yml

## this is the all-in-one, standalone Discourse Docker container template

##

## After making changes to this file, you MUST rebuild

## /var/discourse/launcher rebuild app

##

## BE *VERY* CAREFUL WHEN EDITING!

## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!

## visit http://www.yamllint.com/ to validate this file as needed

templates:

  - "templates/postgres.template.yml"

  - "templates/redis.template.yml"

  - "templates/web.template.yml"

  - "templates/web.ratelimited.template.yml"

## Uncomment these two lines if you wish to add Lets Encrypt (https)

#- "templates/web.ssl.template.yml"

#- "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?

## If you want Discourse to share a port with another webserver like Apache or nginx,

## see https://meta.discourse.org/t/17247 for details

expose:

  - "2045:80" # http

#  - "443:443" # https

params:

  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.

  ## will be set automatically by bootstrap based on detected RAM, or you can override

  db_shared_buffers: "2048MB"

  ## can improve sorting performance, but adds memory usage per-connection

  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)

  #version: tests-passed

env:

  LANG: en_US.UTF-8

  # DISCOURSE_DEFAULT_LOCALE: en

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.

  ## will be set automatically by bootstrap based on detected CPUs, or you can override

  UNICORN_WORKERS: 8

  ## TODO: The domain name this Discourse instance will respond to

  ## Required. Discourse will not work with a bare IP number.

  DISCOURSE_HOSTNAME: "howtodiscuss.com"

  ## Uncomment if you want the container to be started with the same

  ## hostname (-h option) as specified above (default "$hostname-$config")

  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer

  ## on initial signup example 'user1@example.com,user2@example.com'

  DISCOURSE_DEVELOPER_EMAILS: "admin@gmail.com"

  ## TODO: The SMTP mail server used to validate new accounts and send notifications

  # SMTP ADDRESS, username, and password are required

  # WARNING the char '#' in SMTP password can cause problems!

  DISCOURSE_SMTP_ADDRESS: smtp-relay.smtp.com

  DISCOURSE_SMTP_PORT: 587

  DISCOURSE_SMTP_USER_NAME: admin@gmail.com

  DISCOURSE_SMTP_PASSWORD: smtp_password

  #DISCOURSE_SMTP_DOMAIN: discourse.example.com    # (required by some providers)

  #DISCOURSE_NOTIFICATION_EMAIL: nobody@discourse.example.com    # (address to send notifications from)

  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate

  #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

  ## The http or https CDN address for this Discourse instance (configured to pull)

  ## see https://meta.discourse.org/t/14857 for details

  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com

## The Docker container is stateless; all data is stored in /shared

volumes:

  - volume:

      host: /var/discourse/shared/standalone

      guest: /shared

  - volume:

      host: /var/discourse/shared/standalone/log/var-log

      guest: /var/log

## Plugins go here

## see https://meta.discourse.org/t/19157 for details

hooks:

  after_code:

    - exec:

        cd: $home/plugins

        cmd:

          - git clone https://github.com/discourse/docker_manager.git

          - git clone https://github.com/discourse/discourse-sitemap.git

          - git clone https://github.com/discourse/discourse-math.git # custom plugin for math JAX support

          - git clone https://github.com/discourse/discourse-solved.git # custom plugin for question and answers and choosing best answer

          - git clone https://github.com/discourse/discourse-voting.git # custom plugin for voting on topics only

          - git clone https://github.com/discourse/discourse-yearly-review.git # custom plugin for automatically posting a yearly topic of forum activity

          - git clone https://github.com/discourse/discourse-user-notes.git # custom plugin to allow staff to add notes about a user

          - git clone https://github.com/paviliondev/discourse-question-answer.git # custom non official plugin for SO style QnA voting on posts

          - git clone https://github.com/discourse/discourse-adplugin.git # custom official plugin for ads

          - git clone https://github.com/discourse/discourse-whos-online.git # custom plugin to see who is online

          - git clone https://github.com/paviliondev/discourse-topic-previews.git # custom plugin for showing topics previews nicely

          - git clone https://github.com/paviliondev/discourse-ratings.git # custom plugin to allow user rating on topics

          - git clone https://github.com/discourse/discourse-cakeday.git # official plugin for birthdays and join date celebrations

          - git clone https://github.com/discourse/discourse-saved-searches.git # official plugin for saved search notifications

          - git clone https://github.com/paviliondev/discourse-follow.git # custom plugin to allow users to follow each other

          - git clone https://github.com/FaizanZahid/discourse-amp-htd-plugin.git # custom build AMP support for HTD

          - git clone https://github.com/discourse/discourse-assign.git # custom for assigning topics to staff

          - git clone https://github.com/paviliondev/x-discourse-tlp-thumbnail.git # custom plugin for showing thumbnails from CDN images

## Any custom commands to run after building

run:

  - exec: echo "Beginning of custom commands"

  ## If you want to set the 'From' email address for your first registration, uncomment and change:

  ## After getting the first signup email, re-comment the line. It only needs to run once.

  - exec: rails r "SiteSetting.notification_email='noreply@howtodiscuss.com'"

  - exec: echo "End of custom commands"

fzngagan · March 4, 2021, 11:27am

I think you’re running multiple sites on this single instance. Not sure how that would affect the performance issues.

ahmed_khan1 · March 4, 2021, 1:35pm

we are running only this site on this server.

pfaffman · March 4, 2021, 1:48pm

You might bump db_work_mem to 100.

You might have issues with your reverse proxy that is slowing things down.

What does the mini profiler show?

You can also install the prometheus plugin to get some more data.

renato · March 4, 2021, 2:19pm

Have you tried disabling the Who’s Online plugin?

ahmed_khan1 · March 4, 2021, 4:29pm

I have tried but don’t see a big improvement
@pfaffman
here are CPU and memory usage

i have changed unicorn workers to 24
and memory buffers to 4096MB

merefield · March 4, 2021, 4:33pm

Have you looked at the browser end?

Granted this is a snapshot but your CPU utilisation looks tiny. I think you are over speccing.

Memory use is only high because of the number of unicorns. Reduce those if it starts to hit swap badly.

I would definitely look at the data retrieval and JavaScript latencies on the development tools of the browser.

Perceived slowness may be down to the amount of work the browser is doing.

gerhard · March 4, 2021, 4:56pm

From the HTTP headers it looks like you are using Ezoic and Cloudflare. Is that correct? I’m not sure what it does, but that might be another factor for perceived slowness.

My recommendation: Start with a simple setup without any proxies or putting your Discourse behind Cloudflare. Make sure it’s reasonable fast and only then start optimizing by following Enable a CDN for your Discourse or, if you really want to use Cloudflare, Full site CDN acceleration for Discourse.

Faizan_Zahid · March 4, 2021, 8:01pm

That’s correct, only 1 site, which is https://howtodiscuss.com is being served on this server via NGINX. Do you have any ways to boost its speed ?

Topic		Replies	Views
Full site CDN acceleration for Discourse Self-Hosting cdn , explanation	36	18354	September 6, 2023
Discourse with jwilder/nginx-proxy howtos? Installation	12	7816	June 19, 2022
Defaulting to discourse.org CDN for avatars is a privacy and security risk Feature privacy	22	7231	September 28, 2015
Add an offline page to display when Discourse is rebuilding or starting up Sysadmins configuring , how-to , advanced-setup	135	242227	January 5, 2024
Best configurations for speeding up standalone discourse Installation server-resources	26	3898	February 14, 2024

My discourse speed is very slow

Related topics