How to configure Discourse Docker on higher end servers

howto sysadmin
#1

When you install Discourse on an instance with 4GB or more you should consider the following:

Monitor your setup

If you elect to use a higher end setup we strongly recommend you set up monitoring using Newrelic or some other monitoring service. You will need to analyze the results of configuration changes to reach an optimal setup.

Out of the box Discourse Docker ships with 3 web workers

Web workers are served via unicorn, this process is capable of serving one request at a time, you should at least have one worker per CPU. You can increase this number in your template like so

env: 
   # to raise to 6 workers 
   UNICORN_WORKERS: 6

Database configuration is optimised for 2GB of RAM

As guideline you want to dedicate 1/3 of the memory available to postgres to “shared_buffers” to raise this:

params:
  db_shared_buffers: "1GB"

Be sure to install latest Discourse Docker

We update our base templates with various optimisations, be sure to update regularly to ensure you take advantage of it.

14 Likes
Improving Instance Performance (Megatopics, Database Size and Extreme Load)
#2

Thanks @sam I did this changing it to db_shared_buffers"4GB" and UNICORN_WORKERS: 6 and then did a ./launcher rebuild app, how do I now know if those settings took effect? I can’t seem to find anywhere in the admin side to show that info but I am new so quite possibly missing it amongst the many settings screens.

#3

Go to

YOURSITE/admin/upgrade#/processes

Then you can count unicorns and look at memory usage on pg.

1 Like
#4

Thanks @sam will those special options ever be added into the admin settings screen instead of knowing that special url?

I checked and got the following output, which column is showing the mem allocated? VSZ or RSS?

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
discour+   402  0.3  1.4 1774192 355632 ?      Sl   13:25   0:08 unicorn worker[4] -E production -c config/unicorn.conf.rb
discour+   114  0.3  1.3 1817260 332268 ?      Sl   13:25   0:09 sidekiq 4.2.4 discourse [0 of 5 busy]
discour+   332  0.2  1.3 1735260 316636 ?      Sl   13:25   0:06 unicorn worker[3] -E production -c config/unicorn.conf.rb
discour+   253  0.2  1.3 1733216 314304 ?      Sl   13:25   0:06 unicorn worker[2] -E production -c config/unicorn.conf.rb
discour+   172  0.1  1.2 1722972 306632 ?      Sl   13:25   0:05 unicorn worker[1] -E production -c config/unicorn.conf.rb
discour+   518  0.2  1.2 1722972 306600 ?      Sl   13:25   0:05 unicorn worker[5] -E production -c config/unicorn.conf.rb
discour+   138  0.2  1.2 1722976 305936 ?      Sl   13:25   0:05 unicorn worker[0] -E production -c config/unicorn.conf.rb
discour+    87  0.3  0.7 444648 173108 ?       Sl   13:24   0:08 unicorn master -E production -c config/unicorn.conf.rb
postgres    53  0.0  0.5 4471632 136956 ?      S    13:24   0:00 /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main
postgres    82  0.0  0.1 4471736 41056 ?       Ss   13:24   0:00 postgres: 9.5/main: checkpointer process   
postgres    83  0.0  0.1 4471632 37248 ?       Ss   13:24   0:00 postgres: 9.5/main: writer process   
postgres  4108  0.0  0.1 4482208 32208 ?       Ss   14:10   0:00 postgres: 9.5/main: discourse discourse [local] idle
postgres  4210  0.0  0.1 4481128 30308 ?       Ss   14:10   0:00 postgres: 9.5/main: discourse discourse [local] idle
redis       55  0.1  0.1  60908 30100 ?        Sl   13:24   0:03 /usr/bin/redis-server *:6379
postgres  4306  0.1  0.1 4481404 28660 ?       Ss   14:10   0:00 postgres: 9.5/main: discourse discourse [local] idle
postgres  4253  0.0  0.1 4479720 25412 ?       Ss   14:10   0:00 postgres: 9.5/main: discourse discourse [local] idle
postgres  4323  0.0  0.1 4479720 25404 ?       Ss   14:11   0:00 postgres: 9.5/main: discourse discourse [local] idle
postgres  4304  0.0  0.0 4479720 23060 ?       Ss   14:10   0:00 postgres: 9.5/main: discourse discourse [local] idle
postgres    84  0.0  0.0 4471632 21196 ?       Ss   13:24   0:00 postgres: 9.5/main: wal writer process   
discour+  4324  0.0  0.0  47788  7620 ?        R    14:11   0:00 ps aux --sort -rss
root        59  0.0  0.0  50828  6740 ?        S    13:24   0:00 nginx: master process /usr/sbin/nginx
postgres    85  0.0  0.0 4472068 6512 ?        Ss   13:24   0:00 postgres: 9.5/main: autovacuum launcher process   
discour+    52  0.0  0.0  29892  5928 ?        S    13:24   0:01 /bin/bash config/unicorn_launcher -E production -c config/unicorn.conf.rb
discour+  4322  0.0  0.0  20148  5600 ?        S    14:11   0:00 sleep 1
www-data    73  0.0  0.0  51512  4744 ?        S    13:24   0:00 nginx: worker process
www-data    75  0.0  0.0  51512  4684 ?        S    13:24   0:00 nginx: worker process
www-data    77  0.0  0.0  51812  4684 ?        S    13:24   0:00 nginx: worker process
www-data    78  0.0  0.0  51512  4684 ?        S    13:24   0:00 nginx: worker process
www-data    70  0.0  0.0  51512  4620 ?        S    13:24   0:00 nginx: worker process
www-data    72  0.0  0.0  51512  4620 ?        S    13:24   0:00 nginx: worker process
www-data    76  0.0  0.0  51512  4612 ?        S    13:24   0:00 nginx: worker process
www-data    69  0.0  0.0  51512  4560 ?        S    13:24   0:00 nginx: worker process
www-data    74  0.0  0.0  51512  4272 ?        S    13:24   0:00 nginx: worker process
www-data    67  0.0  0.0  51512  4132 ?        S    13:24   0:00 nginx: worker process
www-data    71  0.0  0.0  51512  4132 ?        S    13:24   0:00 nginx: worker process
www-data    68  0.0  0.0  51512  3968 ?        S    13:24   0:00 nginx: worker process
postgres    86  0.0  0.0 150184  3852 ?        Ss   13:24   0:00 postgres: 9.5/main: stats collector process   
root         1  0.0  0.0  21288  3248 ?        Ss+  13:24   0:00 /bin/bash /sbin/boot
syslog      51  0.0  0.0 182664  3120 ?        Sl   13:24   0:00 rsyslogd -n
root        57  0.0  0.0  29300  2984 ?        S    13:24   0:00 cron -f
www-data    79  0.0  0.0  51044  2876 ?        S    13:24   0:00 nginx: cache manager process
root        44  0.0  0.0   4396  1168 ?        S+   13:24   0:00 /usr/bin/runsvdir -P /etc/service
root        45  0.0  0.0   4244   800 ?        Ss   13:24   0:00 runsv nginx
root        46  0.0  0.0   4244   764 ?        Ss   13:24   0:00 runsv unicorn
root        47  0.0  0.0   4244   752 ?        Ss   13:24   0:00 runsv postgres
root        48  0.0  0.0   4244   672 ?        Ss   13:24   0:00 runsv redis
root        49  0.0  0.0   4244   664 ?        Ss   13:24   0:00 runsv cron
root        50  0.0  0.0   4244   660 ?        Ss   13:24   0:00 runsv rsyslog
#5

RSS shows the memory allocated, though its a bit tricky cause some of that is shared between forked processes with master process.

Looks to me like you have 6 unicorns running to me, and looks like lots of memory address space is reserved for pg.

2 Likes
#6

Thanks so it’s probably then allocated the 4GB I specified. Might be nice in a future update to have a simple server settings page inside the admin settings to show this?

#7

Possibly, I would like to at least alert admins if a server is terribly under provisioned.

2 Likes
#8

+1 on this in the GUI, if not as a notification to admins.

You can get an idea of disk space from /admin, something approximating free -h next to it would be another clue.

/admin/upgrade#/processes is another clue as to server load.

Alternately, a report similar to the website performance report, with enough info to warn you if your server is running out of steam.

2 Likes
#9

I have 22Gb allocated to the Proxmox VM I am running it on and only two nginx sites running plus 1 discourse site (proxied through nginx) so I figured I could up that mem allocation substantially to 4Gb. I guess I will see when the site truly goes live and people start using it (hopefully :smile:) how it performs. Lots of free memory to still throw at it when needed but that warning system would indeed be nice instead of waking up to a dead site.