All 8 vCPUs usage on our site is 100%.
Our users get this message often,
Due to extreme load, this is temporarily being shown to everyone as a logged out user would see it.
Our droplet specs are:
CPU-Optimized / 16 GB / 8 vCPUs , Intel Dedicated Digital Ocean Droplet
Traffic:
Online real time users on Google analytics as of writing are approx 70 to 150.
htop snap for CPU and RAM usage is below,
Last 14 days of graph usage from digital ocean:
App.yaml looks like,
db_shared_buffers: “4096MB”
db_work_mem: “400MB”
UNICORN_WORKERS: 16
#- “templates/web.ratelimited.template.yml” #removed this to avoid 429 too many requests errors Remove the ip limits
As you can see, we have commented the ratelimited template because it was giving 429 nginx too many request errors to our normal users despite of very low actual traffic on our site.
CF firewall page looks like,
Launcher log looks like, ./launcher logs app
Prometheus Metrics look like,
# HELP discourse_collector_working Is the master process collector able to collect metrics
# TYPE discourse_collector_working gauge
discourse_collector_working 1
# HELP discourse_collector_rss total memory used by collector process
# TYPE discourse_collector_rss gauge
discourse_collector_rss 46530560
# HELP discourse_collector_metrics_total Total metrics processed by exporter web.
# TYPE discourse_collector_metrics_total counter
discourse_collector_metrics_total 215465
# HELP discourse_collector_sessions_total Total send_metric sessions processed by exporter web.
# TYPE discourse_collector_sessions_total counter
discourse_collector_sessions_total 7499
# HELP discourse_collector_bad_metrics_total Total mis-handled metrics by collector.
# TYPE discourse_collector_bad_metrics_total counter
discourse_collector_bad_metrics_total 0
# HELP discourse_page_views Page views reported by admin dashboard
# TYPE discourse_page_views counter
discourse_page_views{db="default",type="crawler",device="crawler"} 128620
discourse_page_views{db="default",type="anon",device="mobile"} 2840
discourse_page_views{db="default",type="anon",device="desktop"} 4094
discourse_page_views{db="default",type="logged_in",device="mobile"} 55
discourse_page_views{db="default",type="logged_in",device="desktop"} 64
# HELP discourse_http_requests Total HTTP requests from web app
# TYPE discourse_http_requests counter
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="200"} 166429
discourse_http_requests{db="default",api="web",verb="POST",type="background",status="429"} 19628
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="404"} 3945
discourse_http_requests{db="default",api="web",verb="HEAD",type="regular",status="200"} 154
discourse_http_requests{db="default",api="web",verb="POST",type="regular",status="200"} 2232
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="301"} 152
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="304"} 2295
discourse_http_requests{db="default",api="admin",verb="POST",type="regular",status="200"} 97
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="409"} 141
discourse_http_requests{db="default",api="web",verb="POST",type="background",status="200"} 7540
discourse_http_requests{db="default",api="web",verb="POST",type="background",status="-1"} 1985
discourse_http_requests{db="default",api="admin",verb="POST",type="regular",status="422"} 3
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="500"} 1
discourse_http_requests{db="default",api="web",verb="POST",type="regular",status="302"} 3
discourse_http_requests{db="default",api="web",verb="PUT",type="regular",status="200"} 7
discourse_http_requests{db="default",api="web",verb="GET",type="regular",status="302"} 2
# HELP discourse_http_duration_seconds Time spent in HTTP reqs in seconds
# TYPE discourse_http_duration_seconds summary
discourse_http_duration_seconds{controller="other",action="other",quantile="0.99"} 0.942478418815881
discourse_http_duration_seconds{controller="other",action="other",quantile="0.9"} 0.05924026109278202
discourse_http_duration_seconds{controller="other",action="other",quantile="0.5"} 0.000657188706099987
discourse_http_duration_seconds{controller="other",action="other",quantile="0.1"} 0.0004582637920975685
discourse_http_duration_seconds{controller="other",action="other",quantile="0.01"} 0.0004117507487535477
discourse_http_duration_seconds_sum{controller="other",action="other"} 3942.3923240867443
discourse_http_duration_seconds_count{controller="other",action="other"} 58870
discourse_http_duration_seconds{controller="topics",action="show",quantile="0.99"} 4.58280345518142
discourse_http_duration_seconds{controller="topics",action="show",quantile="0.9"} 2.827586194965988
discourse_http_duration_seconds{controller="topics",action="show",quantile="0.5"} 0.01292009698227048
discourse_http_duration_seconds{controller="topics",action="show",quantile="0.1"} 0.002048521768301725
discourse_http_duration_seconds{controller="topics",action="show",quantile="0.01"} 0.001819557975977659
discourse_http_duration_seconds_sum{controller="topics",action="show"} 128537.17967935232
discourse_http_duration_seconds_count{controller="topics",action="show"} 143926
discourse_http_duration_seconds{controller="list",action="latest",quantile="0.99"} 5.726923605892807
discourse_http_duration_seconds{controller="list",action="latest",quantile="0.9"} 5.168957637157291
discourse_http_duration_seconds{controller="list",action="latest",quantile="0.5"} 0.009740645997226238
discourse_http_duration_seconds{controller="list",action="latest",quantile="0.1"} 0.003163302782922983
discourse_http_duration_seconds{controller="list",action="latest",quantile="0.01"} 0.002744518686085939
discourse_http_duration_seconds_sum{controller="list",action="latest"} 3052.3646773528308
discourse_http_duration_seconds_count{controller="list",action="latest"} 1676
discourse_http_duration_seconds{controller="users",action="show",quantile="0.99"} 0.1768659572117031
discourse_http_duration_seconds{controller="users",action="show",quantile="0.9"} 0.1768659572117031
discourse_http_duration_seconds{controller="users",action="show",quantile="0.5"} 0.1768659572117031
discourse_http_duration_seconds{controller="users",action="show",quantile="0.1"} 0.1768659572117031
discourse_http_duration_seconds{controller="users",action="show",quantile="0.01"} 0.1768659572117031
discourse_http_duration_seconds_sum{controller="users",action="show"} 22.969309298321605
discourse_http_duration_seconds_count{controller="users",action="show"} 142
# HELP discourse_http_redis_duration_seconds Time spent in HTTP reqs in redis seconds
# TYPE discourse_http_redis_duration_seconds summary
discourse_http_redis_duration_seconds{controller="other",action="other",quantile="0.99"} 0.02405449468642473
discourse_http_redis_duration_seconds{controller="other",action="other",quantile="0.9"} 0.0092271463945508
discourse_http_redis_duration_seconds{controller="other",action="other",quantile="0.5"} 0.0
discourse_http_redis_duration_seconds{controller="other",action="other",quantile="0.1"} 0.0
discourse_http_redis_duration_seconds{controller="other",action="other",quantile="0.01"} 0.0
discourse_http_redis_duration_seconds_sum{controller="other",action="other"} 177.52055060770363
discourse_http_redis_duration_seconds_count{controller="other",action="other"} 58870
discourse_http_redis_duration_seconds{controller="topics",action="show",quantile="0.99"} 0.064909300301224
discourse_http_redis_duration_seconds{controller="topics",action="show",quantile="0.9"} 0.03797017782926559
discourse_http_redis_duration_seconds{controller="topics",action="show",quantile="0.5"} 0.007125054951757193
discourse_http_redis_duration_seconds{controller="topics",action="show",quantile="0.1"} 0.0005201729945838451
discourse_http_redis_duration_seconds{controller="topics",action="show",quantile="0.01"} 0.0004380182363092899
discourse_http_redis_duration_seconds_sum{controller="topics",action="show"} 1814.5181179852225
discourse_http_redis_duration_seconds_count{controller="topics",action="show"} 143924
discourse_http_redis_duration_seconds{controller="list",action="latest",quantile="0.99"} 0.03446856886148453
discourse_http_redis_duration_seconds{controller="list",action="latest",quantile="0.9"} 0.02759678149595857
discourse_http_redis_duration_seconds{controller="list",action="latest",quantile="0.5"} 0.004360581748187542
discourse_http_redis_duration_seconds{controller="list",action="latest",quantile="0.1"} 0.001454399898648262
discourse_http_redis_duration_seconds{controller="list",action="latest",quantile="0.01"} 0.00108026061207056
discourse_http_redis_duration_seconds_sum{controller="list",action="latest"} 20.32011368824169
discourse_http_redis_duration_seconds_count{controller="list",action="latest"} 1676
discourse_http_redis_duration_seconds{controller="users",action="show",quantile="0.99"} 0.01075387001037598
discourse_http_redis_duration_seconds{controller="users",action="show",quantile="0.9"} 0.01075387001037598
discourse_http_redis_duration_seconds{controller="users",action="show",quantile="0.5"} 0.01075387001037598
discourse_http_redis_duration_seconds{controller="users",action="show",quantile="0.1"} 0.01075387001037598
discourse_http_redis_duration_seconds{controller="users",action="show",quantile="0.01"} 0.01075387001037598
discourse_http_redis_duration_seconds_sum{controller="users",action="show"} 1.3520850692875683
discourse_http_redis_duration_seconds_count{controller="users",action="show"} 142
# HELP discourse_http_sql_duration_seconds Time spent in HTTP reqs in SQL in seconds
# TYPE discourse_http_sql_duration_seconds summary
discourse_http_sql_duration_seconds{controller="other",action="other",quantile="0.99"} 0.1874205535277724
discourse_http_sql_duration_seconds{controller="other",action="other",quantile="0.9"} 0.009177460800856352
discourse_http_sql_duration_seconds{controller="other",action="other",quantile="0.5"} 0.0
discourse_http_sql_duration_seconds{controller="other",action="other",quantile="0.1"} 0.0
discourse_http_sql_duration_seconds{controller="other",action="other",quantile="0.01"} 0.0
discourse_http_sql_duration_seconds_sum{controller="other",action="other"} 375.05954352673143
discourse_http_sql_duration_seconds_count{controller="other",action="other"} 58870
discourse_http_sql_duration_seconds{controller="topics",action="show",quantile="0.99"} 0.4393073697574437
discourse_http_sql_duration_seconds{controller="topics",action="show",quantile="0.9"} 0.2883010846562684
discourse_http_sql_duration_seconds{controller="topics",action="show",quantile="0.5"} 0.0
discourse_http_sql_duration_seconds{controller="topics",action="show",quantile="0.1"} 0.0
discourse_http_sql_duration_seconds{controller="topics",action="show",quantile="0.01"} 0.0
discourse_http_sql_duration_seconds_sum{controller="topics",action="show"} 13847.843351929914
discourse_http_sql_duration_seconds_count{controller="topics",action="show"} 143926
discourse_http_sql_duration_seconds{controller="list",action="latest",quantile="0.99"} 0.6733640311285853
discourse_http_sql_duration_seconds{controller="list",action="latest",quantile="0.9"} 0.5781291825696826
discourse_http_sql_duration_seconds{controller="list",action="latest",quantile="0.5"} 0.0
discourse_http_sql_duration_seconds{controller="list",action="latest",quantile="0.1"} 0.0
discourse_http_sql_duration_seconds{controller="list",action="latest",quantile="0.01"} 0.0
discourse_http_sql_duration_seconds_sum{controller="list",action="latest"} 324.1082574711181
discourse_http_sql_duration_seconds_count{controller="list",action="latest"} 1676
discourse_http_sql_duration_seconds{controller="users",action="show",quantile="0.99"} 0.07832226948812604
discourse_http_sql_duration_seconds{controller="users",action="show",quantile="0.9"} 0.07832226948812604
discourse_http_sql_duration_seconds{controller="users",action="show",quantile="0.5"} 0.07832226948812604
discourse_http_sql_duration_seconds{controller="users",action="show",quantile="0.1"} 0.07832226948812604
discourse_http_sql_duration_seconds{controller="users",action="show",quantile="0.01"} 0.07832226948812604
discourse_http_sql_duration_seconds_sum{controller="users",action="show"} 9.40852727741003
discourse_http_sql_duration_seconds_count{controller="users",action="show"} 142
# HELP discourse_http_net_duration_seconds Time spent in external network requests
# TYPE discourse_http_net_duration_seconds summary
discourse_http_net_duration_seconds{controller="other",action="other",quantile="0.99"} 0.0
discourse_http_net_duration_seconds{controller="other",action="other",quantile="0.9"} 0.0
discourse_http_net_duration_seconds{controller="other",action="other",quantile="0.5"} 0.0
discourse_http_net_duration_seconds{controller="other",action="other",quantile="0.1"} 0.0
discourse_http_net_duration_seconds{controller="other",action="other",quantile="0.01"} 0.0
discourse_http_net_duration_seconds_sum{controller="other",action="other"} 349.2784005794674
discourse_http_net_duration_seconds_count{controller="other",action="other"} 58870
discourse_http_net_duration_seconds{controller="topics",action="show",quantile="0.99"} 0.0
discourse_http_net_duration_seconds{controller="topics",action="show",quantile="0.9"} 0.0
discourse_http_net_duration_seconds{controller="topics",action="show",quantile="0.5"} 0.0
discourse_http_net_duration_seconds{controller="topics",action="show",quantile="0.1"} 0.0
discourse_http_net_duration_seconds{controller="topics",action="show",quantile="0.01"} 0.0
discourse_http_net_duration_seconds_sum{controller="topics",action="show"} 0.0
discourse_http_net_duration_seconds_count{controller="topics",action="show"} 143926
discourse_http_net_duration_seconds{controller="list",action="latest",quantile="0.99"} 0.0
discourse_http_net_duration_seconds{controller="list",action="latest",quantile="0.9"} 0.0
discourse_http_net_duration_seconds{controller="list",action="latest",quantile="0.5"} 0.0
discourse_http_net_duration_seconds{controller="list",action="latest",quantile="0.1"} 0.0
discourse_http_net_duration_seconds{controller="list",action="latest",quantile="0.01"} 0.0
discourse_http_net_duration_seconds_sum{controller="list",action="latest"} 0.0
discourse_http_net_duration_seconds_count{controller="list",action="latest"} 1676
discourse_http_net_duration_seconds{controller="users",action="show",quantile="0.99"} 0.0
discourse_http_net_duration_seconds{controller="users",action="show",quantile="0.9"} 0.0
discourse_http_net_duration_seconds{controller="users",action="show",quantile="0.5"} 0.0
discourse_http_net_duration_seconds{controller="users",action="show",quantile="0.1"} 0.0
discourse_http_net_duration_seconds{controller="users",action="show",quantile="0.01"} 0.0
discourse_http_net_duration_seconds_sum{controller="users",action="show"} 0.0
discourse_http_net_duration_seconds_count{controller="users",action="show"} 142
# HELP discourse_http_queue_duration_seconds Time spent queueing requests between NGINX and Ruby
# TYPE discourse_http_queue_duration_seconds summary
discourse_http_queue_duration_seconds{controller="other",action="other",quantile="0.99"} 7.055693626403809
discourse_http_queue_duration_seconds{controller="other",action="other",quantile="0.9"} 5.77228569984436
discourse_http_queue_duration_seconds{controller="other",action="other",quantile="0.5"} 3.628639221191406
discourse_http_queue_duration_seconds{controller="other",action="other",quantile="0.1"} 1.691039323806763
discourse_http_queue_duration_seconds{controller="other",action="other",quantile="0.01"} 0.1517226696014404
discourse_http_queue_duration_seconds_sum{controller="other",action="other"} 128979.19509196281
discourse_http_queue_duration_seconds_count{controller="other",action="other"} 58870
discourse_http_queue_duration_seconds{controller="topics",action="show",quantile="0.99"} 6.773485660552979
discourse_http_queue_duration_seconds{controller="topics",action="show",quantile="0.9"} 5.540528535842896
discourse_http_queue_duration_seconds{controller="topics",action="show",quantile="0.5"} 3.386407375335693
discourse_http_queue_duration_seconds{controller="topics",action="show",quantile="0.1"} 1.529008388519287
discourse_http_queue_duration_seconds{controller="topics",action="show",quantile="0.01"} 0.3203737735748291
discourse_http_queue_duration_seconds_sum{controller="topics",action="show"} 358491.9590744972
discourse_http_queue_duration_seconds_count{controller="topics",action="show"} 143926
discourse_http_queue_duration_seconds{controller="list",action="latest",quantile="0.99"} 7.470051765441895
discourse_http_queue_duration_seconds{controller="list",action="latest",quantile="0.9"} 5.970950841903687
discourse_http_queue_duration_seconds{controller="list",action="latest",quantile="0.5"} 3.233286619186401
discourse_http_queue_duration_seconds{controller="list",action="latest",quantile="0.1"} 1.421695709228516
discourse_http_queue_duration_seconds{controller="list",action="latest",quantile="0.01"} 0.4568729400634766
discourse_http_queue_duration_seconds_sum{controller="list",action="latest"} 5290.475468635559
discourse_http_queue_duration_seconds_count{controller="list",action="latest"} 1676
discourse_http_queue_duration_seconds{controller="users",action="show",quantile="0.99"} 5.078022718429565
discourse_http_queue_duration_seconds{controller="users",action="show",quantile="0.9"} 5.078022718429565
discourse_http_queue_duration_seconds{controller="users",action="show",quantile="0.5"} 5.078022718429565
discourse_http_queue_duration_seconds{controller="users",action="show",quantile="0.1"} 5.078022718429565
discourse_http_queue_duration_seconds{controller="users",action="show",quantile="0.01"} 5.078022718429565
discourse_http_queue_duration_seconds_sum{controller="users",action="show"} 381.56648421287537
discourse_http_queue_duration_seconds_count{controller="users",action="show"} 142
# HELP discourse_http_forced_anon_count Total count of logged in requests forced into anonymous mode
# TYPE discourse_http_forced_anon_count counter
discourse_http_forced_anon_count{db="default",api="web",verb="GET",type="regular",status="200"} 38907
discourse_http_forced_anon_count{db="default",api="web",verb="GET",type="regular",status="404"} 1236
# HELP discourse_http_sql_calls_per_request How many SQL statements ran per request
# TYPE discourse_http_sql_calls_per_request gauge
discourse_http_sql_calls_per_request{controller="users",action="show",logged_in="false"} 30
discourse_http_sql_calls_per_request{controller="list",action="latest",logged_in="true"} 271
discourse_http_sql_calls_per_request{controller="topics",action="show",logged_in="true"} 164
discourse_http_sql_calls_per_request{controller="users",action="show",logged_in="true"} 98
discourse_http_sql_calls_per_request{controller="list",action="latest",logged_in="false"} 223
discourse_http_sql_calls_per_request{controller="other",action="other",logged_in="false"} 6
discourse_http_sql_calls_per_request{controller="topics",action="show",logged_in="false"} 136
# HELP discourse_http_anon_cache_store How many a payload is stored in redis for anonymous cache
# TYPE discourse_http_anon_cache_store counter
discourse_http_anon_cache_store{controller="topics",action="show"} 21358
discourse_http_anon_cache_store{controller="other",action="other"} 260
discourse_http_anon_cache_store{controller="list",action="latest"} 286
# HELP discourse_http_anon_cache_hit How many a payload from redis is used for anonymous cache
# TYPE discourse_http_anon_cache_hit counter
discourse_http_anon_cache_hit{controller="other",action="other"} 7507
discourse_http_anon_cache_hit{controller="topics",action="show"} 82761
discourse_http_anon_cache_hit{controller="list",action="latest"} 929
# HELP discourse_heap_free_slots Free ruby heap slots
# TYPE discourse_heap_free_slots gauge
discourse_heap_free_slots{type="sidekiq",pid="131"} 12166
discourse_heap_free_slots{type="web",pid="163"} 119712
discourse_heap_free_slots{type="web",pid="183"} 69322
discourse_heap_free_slots{type="web",pid="454"} 31247
discourse_heap_free_slots{type="web",pid="201"} 162260
discourse_heap_free_slots{type="web",pid="503"} 83322
discourse_heap_free_slots{type="web",pid="308"} 72679
discourse_heap_free_slots{type="web",pid="328"} 79991
discourse_heap_free_slots{type="web",pid="396"} 204322
discourse_heap_free_slots{type="web",pid="219"} 221194
discourse_heap_free_slots{type="web",pid="374"} 122163
discourse_heap_free_slots{type="web",pid="270"} 43722
discourse_heap_free_slots{type="web",pid="418"} 155036
discourse_heap_free_slots{type="web",pid="434"} 158153
discourse_heap_free_slots{type="web",pid="479"} 116781
discourse_heap_free_slots{type="web",pid="350"} 167763
discourse_heap_free_slots{type="web",pid="287"} 95786
# HELP discourse_heap_live_slots Used ruby heap slots
# TYPE discourse_heap_live_slots gauge
discourse_heap_live_slots{type="sidekiq",pid="131"} 883728
discourse_heap_live_slots{type="web",pid="163"} 816121
discourse_heap_live_slots{type="web",pid="183"} 866514
discourse_heap_live_slots{type="web",pid="454"} 904588
discourse_heap_live_slots{type="web",pid="201"} 773576
discourse_heap_live_slots{type="web",pid="503"} 852518
discourse_heap_live_slots{type="web",pid="308"} 903104
discourse_heap_live_slots{type="web",pid="328"} 855035
discourse_heap_live_slots{type="web",pid="396"} 731505
discourse_heap_live_slots{type="web",pid="219"} 714638
discourse_heap_live_slots{type="web",pid="374"} 810818
discourse_heap_live_slots{type="web",pid="270"} 892104
discourse_heap_live_slots{type="web",pid="418"} 780793
discourse_heap_live_slots{type="web",pid="434"} 777678
discourse_heap_live_slots{type="web",pid="479"} 819045
discourse_heap_live_slots{type="web",pid="350"} 768077
discourse_heap_live_slots{type="web",pid="287"} 840051
# HELP discourse_v8_heap_size Total JavaScript V8 heap size (bytes)
# TYPE discourse_v8_heap_size gauge
discourse_v8_heap_size{type="sidekiq",pid="131"} 16392192
discourse_v8_heap_size{type="web",pid="163"} 15900672
discourse_v8_heap_size{type="web",pid="183"} 16424960
discourse_v8_heap_size{type="web",pid="454"} 17702912
discourse_v8_heap_size{type="web",pid="201"} 16433152
discourse_v8_heap_size{type="web",pid="503"} 17440768
discourse_v8_heap_size{type="web",pid="308"} 17178624
discourse_v8_heap_size{type="web",pid="328"} 16424960
discourse_v8_heap_size{type="web",pid="396"} 16162816
discourse_v8_heap_size{type="web",pid="219"} 16171008
discourse_v8_heap_size{type="web",pid="374"} 16171008
discourse_v8_heap_size{type="web",pid="270"} 17219584
discourse_v8_heap_size{type="web",pid="418"} 16957440
discourse_v8_heap_size{type="web",pid="434"} 15908864
discourse_v8_heap_size{type="web",pid="479"} 16654336
discourse_v8_heap_size{type="web",pid="350"} 16949248
discourse_v8_heap_size{type="web",pid="287"} 16654336
# HELP discourse_v8_used_heap_size Total used JavaScript V8 heap size (bytes)
# TYPE discourse_v8_used_heap_size gauge
discourse_v8_used_heap_size{type="sidekiq",pid="131"} 12932148
discourse_v8_used_heap_size{type="web",pid="163"} 12252448
discourse_v8_used_heap_size{type="web",pid="183"} 12526820
discourse_v8_used_heap_size{type="web",pid="454"} 12993364
discourse_v8_used_heap_size{type="web",pid="201"} 12462852
discourse_v8_used_heap_size{type="web",pid="503"} 12626676
discourse_v8_used_heap_size{type="web",pid="308"} 12702604
discourse_v8_used_heap_size{type="web",pid="328"} 12126824
discourse_v8_used_heap_size{type="web",pid="396"} 12269764
discourse_v8_used_heap_size{type="web",pid="219"} 12293912
discourse_v8_used_heap_size{type="web",pid="374"} 12210560
discourse_v8_used_heap_size{type="web",pid="270"} 12628080
discourse_v8_used_heap_size{type="web",pid="418"} 12538324
discourse_v8_used_heap_size{type="web",pid="434"} 12179620
discourse_v8_used_heap_size{type="web",pid="479"} 12715960
discourse_v8_used_heap_size{type="web",pid="350"} 12362024
discourse_v8_used_heap_size{type="web",pid="287"} 12654196
# HELP discourse_v8_physical_size Physical size consumed by V8 heaps
# TYPE discourse_v8_physical_size gauge
discourse_v8_physical_size{type="sidekiq",pid="131"} 14118508
discourse_v8_physical_size{type="web",pid="163"} 13315992
discourse_v8_physical_size{type="web",pid="183"} 13639620
discourse_v8_physical_size{type="web",pid="454"} 14201860
discourse_v8_physical_size{type="web",pid="201"} 13543736
discourse_v8_physical_size{type="web",pid="503"} 13779544
discourse_v8_physical_size{type="web",pid="308"} 13836520
discourse_v8_physical_size{type="web",pid="328"} 13250748
discourse_v8_physical_size{type="web",pid="396"} 13379408
discourse_v8_physical_size{type="web",pid="219"} 13401800
discourse_v8_physical_size{type="web",pid="374"} 13285780
discourse_v8_physical_size{type="web",pid="270"} 13776112
discourse_v8_physical_size{type="web",pid="418"} 13634224
discourse_v8_physical_size{type="web",pid="434"} 13221264
discourse_v8_physical_size{type="web",pid="479"} 13876884
discourse_v8_physical_size{type="web",pid="350"} 13471300
discourse_v8_physical_size{type="web",pid="287"} 13738052
# HELP discourse_v8_heap_count Number of V8 contexts running
# TYPE discourse_v8_heap_count gauge
discourse_v8_heap_count{type="sidekiq",pid="131"} 2
discourse_v8_heap_count{type="web",pid="163"} 2
discourse_v8_heap_count{type="web",pid="183"} 2
discourse_v8_heap_count{type="web",pid="454"} 2
discourse_v8_heap_count{type="web",pid="201"} 2
discourse_v8_heap_count{type="web",pid="503"} 2
discourse_v8_heap_count{type="web",pid="308"} 2
discourse_v8_heap_count{type="web",pid="328"} 2
discourse_v8_heap_count{type="web",pid="396"} 2
discourse_v8_heap_count{type="web",pid="219"} 2
discourse_v8_heap_count{type="web",pid="374"} 2
discourse_v8_heap_count{type="web",pid="270"} 2
discourse_v8_heap_count{type="web",pid="418"} 2
discourse_v8_heap_count{type="web",pid="434"} 2
discourse_v8_heap_count{type="web",pid="479"} 2
discourse_v8_heap_count{type="web",pid="350"} 2
discourse_v8_heap_count{type="web",pid="287"} 2
# HELP discourse_rss Total RSS used by process
# TYPE discourse_rss gauge
discourse_rss{type="sidekiq",pid="131"} 412172288
discourse_rss{type="web",pid="163"} 481796096
discourse_rss{type="web",pid="183"} 485736448
discourse_rss{type="web",pid="454"} 432590848
discourse_rss{type="web",pid="201"} 408793088
discourse_rss{type="web",pid="503"} 400375808
discourse_rss{type="web",pid="308"} 444911616
discourse_rss{type="web",pid="328"} 418308096
discourse_rss{type="web",pid="396"} 413487104
discourse_rss{type="web",pid="219"} 400482304
discourse_rss{type="web",pid="374"} 440221696
discourse_rss{type="web",pid="270"} 501215232
discourse_rss{type="web",pid="418"} 479678464
discourse_rss{type="web",pid="434"} 431116288
discourse_rss{type="web",pid="479"} 428220416
discourse_rss{type="web",pid="350"} 430444544
discourse_rss{type="web",pid="287"} 428658688
# HELP discourse_thread_count Total number of active threads per process
# TYPE discourse_thread_count gauge
discourse_thread_count{type="sidekiq",pid="131"} 22
discourse_thread_count{type="web",pid="163"} 8
discourse_thread_count{type="web",pid="183"} 8
discourse_thread_count{type="web",pid="454"} 8
discourse_thread_count{type="web",pid="201"} 8
discourse_thread_count{type="web",pid="503"} 8
discourse_thread_count{type="web",pid="308"} 8
discourse_thread_count{type="web",pid="328"} 8
discourse_thread_count{type="web",pid="396"} 8
discourse_thread_count{type="web",pid="219"} 8
discourse_thread_count{type="web",pid="374"} 8
discourse_thread_count{type="web",pid="270"} 8
discourse_thread_count{type="web",pid="418"} 8
discourse_thread_count{type="web",pid="434"} 8
discourse_thread_count{type="web",pid="479"} 8
discourse_thread_count{type="web",pid="350"} 8
discourse_thread_count{type="web",pid="287"} 8
# HELP discourse_deferred_jobs_queued Number of jobs queued in the deferred job queue
# TYPE discourse_deferred_jobs_queued gauge
discourse_deferred_jobs_queued{type="sidekiq",pid="131"} 0
discourse_deferred_jobs_queued{type="web",pid="163"} 0
discourse_deferred_jobs_queued{type="web",pid="183"} 0
discourse_deferred_jobs_queued{type="web",pid="454"} 0
discourse_deferred_jobs_queued{type="web",pid="201"} 0
discourse_deferred_jobs_queued{type="web",pid="503"} 0
discourse_deferred_jobs_queued{type="web",pid="308"} 0
discourse_deferred_jobs_queued{type="web",pid="328"} 0
discourse_deferred_jobs_queued{type="web",pid="396"} 0
discourse_deferred_jobs_queued{type="web",pid="219"} 0
discourse_deferred_jobs_queued{type="web",pid="374"} 0
discourse_deferred_jobs_queued{type="web",pid="270"} 0
discourse_deferred_jobs_queued{type="web",pid="418"} 0
discourse_deferred_jobs_queued{type="web",pid="434"} 0
discourse_deferred_jobs_queued{type="web",pid="479"} 0
discourse_deferred_jobs_queued{type="web",pid="350"} 0
discourse_deferred_jobs_queued{type="web",pid="287"} 0
# HELP discourse_active_record_connections_count Total number of connections in ActiveRecord's connection pools
# TYPE discourse_active_record_connections_count gauge
discourse_active_record_connections_count{type="sidekiq",pid="131",status="busy"} 2
discourse_active_record_connections_count{type="sidekiq",pid="131",status="dead"} 0
discourse_active_record_connections_count{type="sidekiq",pid="131",status="idle"} 4
discourse_active_record_connections_count{type="sidekiq",pid="131",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="163",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="163",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="163",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="163",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="183",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="183",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="183",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="183",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="454",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="454",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="454",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="454",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="201",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="201",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="201",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="201",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="503",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="503",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="503",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="503",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="308",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="308",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="308",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="308",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="328",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="328",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="328",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="328",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="396",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="396",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="396",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="396",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="219",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="219",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="219",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="219",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="374",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="374",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="374",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="374",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="270",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="270",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="270",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="270",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="418",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="418",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="418",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="418",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="434",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="434",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="434",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="434",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="479",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="479",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="479",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="479",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="350",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="350",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="350",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="350",status="waiting"} 0
discourse_active_record_connections_count{type="web",pid="287",status="busy"} 1
discourse_active_record_connections_count{type="web",pid="287",status="dead"} 0
discourse_active_record_connections_count{type="web",pid="287",status="idle"} 1
discourse_active_record_connections_count{type="web",pid="287",status="waiting"} 0
# HELP discourse_active_record_failover_count Count of ActiveRecord databases in a failover state
# TYPE discourse_active_record_failover_count gauge
discourse_active_record_failover_count{type="sidekiq",pid="131"} 0
discourse_active_record_failover_count{type="web",pid="163"} 0
discourse_active_record_failover_count{type="web",pid="183"} 0
discourse_active_record_failover_count{type="web",pid="454"} 0
discourse_active_record_failover_count{type="web",pid="201"} 0
discourse_active_record_failover_count{type="web",pid="503"} 0
discourse_active_record_failover_count{type="web",pid="308"} 0
discourse_active_record_failover_count{type="web",pid="328"} 0
discourse_active_record_failover_count{type="web",pid="396"} 0
discourse_active_record_failover_count{type="web",pid="219"} 0
discourse_active_record_failover_count{type="web",pid="374"} 0
discourse_active_record_failover_count{type="web",pid="270"} 0
discourse_active_record_failover_count{type="web",pid="418"} 0
discourse_active_record_failover_count{type="web",pid="434"} 0
discourse_active_record_failover_count{type="web",pid="479"} 0
discourse_active_record_failover_count{type="web",pid="350"} 0
discourse_active_record_failover_count{type="web",pid="287"} 0
# HELP discourse_redis_failover_count Count of Redis servers in a failover state
# TYPE discourse_redis_failover_count gauge
discourse_redis_failover_count{type="sidekiq",pid="131"} 0
discourse_redis_failover_count{type="web",pid="163"} 0
discourse_redis_failover_count{type="web",pid="183"} 0
discourse_redis_failover_count{type="web",pid="454"} 0
discourse_redis_failover_count{type="web",pid="201"} 0
discourse_redis_failover_count{type="web",pid="503"} 0
discourse_redis_failover_count{type="web",pid="308"} 0
discourse_redis_failover_count{type="web",pid="328"} 0
discourse_redis_failover_count{type="web",pid="396"} 0
discourse_redis_failover_count{type="web",pid="219"} 0
discourse_redis_failover_count{type="web",pid="374"} 0
discourse_redis_failover_count{type="web",pid="270"} 0
discourse_redis_failover_count{type="web",pid="418"} 0
discourse_redis_failover_count{type="web",pid="434"} 0
discourse_redis_failover_count{type="web",pid="479"} 0
discourse_redis_failover_count{type="web",pid="350"} 0
discourse_redis_failover_count{type="web",pid="287"} 0
# HELP discourse_major_gc_count Major GC operations by process
# TYPE discourse_major_gc_count counter
discourse_major_gc_count{type="sidekiq",pid="131"} 39
discourse_major_gc_count{type="web",pid="163"} 902
discourse_major_gc_count{type="web",pid="183"} 848
discourse_major_gc_count{type="web",pid="454"} 881
discourse_major_gc_count{type="web",pid="201"} 896
discourse_major_gc_count{type="web",pid="503"} 849
discourse_major_gc_count{type="web",pid="308"} 878
discourse_major_gc_count{type="web",pid="328"} 888
discourse_major_gc_count{type="web",pid="396"} 882
discourse_major_gc_count{type="web",pid="219"} 897
discourse_major_gc_count{type="web",pid="374"} 919
discourse_major_gc_count{type="web",pid="270"} 883
discourse_major_gc_count{type="web",pid="418"} 921
discourse_major_gc_count{type="web",pid="434"} 853
discourse_major_gc_count{type="web",pid="479"} 923
discourse_major_gc_count{type="web",pid="350"} 909
discourse_major_gc_count{type="web",pid="287"} 823
# HELP discourse_minor_gc_count Minor GC operations by process
# TYPE discourse_minor_gc_count counter
discourse_minor_gc_count{type="sidekiq",pid="131"} 251
discourse_minor_gc_count{type="web",pid="163"} 8220
discourse_minor_gc_count{type="web",pid="183"} 8171
discourse_minor_gc_count{type="web",pid="454"} 8435
discourse_minor_gc_count{type="web",pid="201"} 8249
discourse_minor_gc_count{type="web",pid="503"} 8252
discourse_minor_gc_count{type="web",pid="308"} 8356
discourse_minor_gc_count{type="web",pid="328"} 8276
discourse_minor_gc_count{type="web",pid="396"} 8377
discourse_minor_gc_count{type="web",pid="219"} 8311
discourse_minor_gc_count{type="web",pid="374"} 8521
discourse_minor_gc_count{type="web",pid="270"} 8256
discourse_minor_gc_count{type="web",pid="418"} 8503
discourse_minor_gc_count{type="web",pid="434"} 8332
discourse_minor_gc_count{type="web",pid="479"} 8577
discourse_minor_gc_count{type="web",pid="350"} 8480
discourse_minor_gc_count{type="web",pid="287"} 8241
# HELP discourse_total_allocated_objects Total number of allocateds objects by process
# TYPE discourse_total_allocated_objects counter
discourse_total_allocated_objects{type="sidekiq",pid="131"} 30060560
discourse_total_allocated_objects{type="web",pid="163"} 985232557
discourse_total_allocated_objects{type="web",pid="183"} 983805640
discourse_total_allocated_objects{type="web",pid="454"} 1008867616
discourse_total_allocated_objects{type="web",pid="201"} 988967755
discourse_total_allocated_objects{type="web",pid="503"} 998885290
discourse_total_allocated_objects{type="web",pid="308"} 1004069224
discourse_total_allocated_objects{type="web",pid="328"} 996554004
discourse_total_allocated_objects{type="web",pid="396"} 1013631939
discourse_total_allocated_objects{type="web",pid="219"} 999330059
discourse_total_allocated_objects{type="web",pid="374"} 1016924963
discourse_total_allocated_objects{type="web",pid="270"} 989042711
discourse_total_allocated_objects{type="web",pid="418"} 1024175277
discourse_total_allocated_objects{type="web",pid="434"} 1004407864
discourse_total_allocated_objects{type="web",pid="479"} 1025782126
discourse_total_allocated_objects{type="web",pid="350"} 1013371770
discourse_total_allocated_objects{type="web",pid="287"} 1003799360
# HELP discourse_scheduled_job_duration_seconds Total time spent in scheduled jobs
# TYPE discourse_scheduled_job_duration_seconds counter
discourse_scheduled_job_duration_seconds{job_name="Jobs::PendingUsersReminder"} 0.644000000000000011e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::WhosOnlineGoingOffline"} 0.5267999999999998071e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::PresenceChannelAutoLeave"} 0.6388000000000000119e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::ProcessBadgeBacklog"} 0.25297000000000000432e2
discourse_scheduled_job_duration_seconds{job_name="Jobs::TopicTimerEnqueuer"} 0.5947000000000000053e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::Heartbeat"} 0.1635000000000000038e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::ProcessShelvedNotifications"} 0.1221000000000000029e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::EnqueueOnceoffs"} 0.4557000000000000043e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::BookmarkReminderNotifications"} 0.1019000000000000007e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::CleanUpUploads"} 0.138000000000000002e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::PollMailbox"} 0.851000000000000004e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::TopRefreshToday"} 13.684000000000001
discourse_scheduled_job_duration_seconds{job_name="Jobs::DestroyOldDeletionStubs"} 0.272400000000000003e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::ActivationReminderEmails"} 0.096
discourse_scheduled_job_duration_seconds{job_name="Jobs::UnsilenceUsers"} 0.8790000000000002
discourse_scheduled_job_duration_seconds{job_name="Jobs::PeriodicalUpdates"} 0.1145550000000000045e3
discourse_scheduled_job_duration_seconds{job_name="Jobs::MigrateUploadScheme"} 0.388000000000000004e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::AboutStats"} 2.139
discourse_scheduled_job_duration_seconds{job_name="Jobs::CheckOutOfDateThemes"} 1.68
discourse_scheduled_job_duration_seconds{job_name="Jobs::EnqueueDigestEmails"} 0.156000000000000008e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::DirectoryRefreshDaily"} 0.43910000000000002e1
discourse_scheduled_job_duration_seconds{job_name="Jobs::DashboardStats"} 0.153000000000000009e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::PendingReviewablesReminder"} 0.060000000000000005
discourse_scheduled_job_duration_seconds{job_name="Jobs::CreateMissingAvatars"} 0.200000000000000005e0
discourse_scheduled_job_duration_seconds{job_name="Jobs::UpdateAnimatedUploads"} 0.133
discourse_scheduled_job_duration_seconds{job_name="Jobs::EnqueueSuspectUsers"} 0.12100000000000001
discourse_scheduled_job_duration_seconds{job_name="Jobs::UpdateSitemaps"} 3.831
discourse_scheduled_job_duration_seconds{job_name="Jobs::PendingQueuedPostsReminder"} 2.649
discourse_scheduled_job_duration_seconds{job_name="Jobs::ReindexSearch"} 1.431
discourse_scheduled_job_duration_seconds{job_name="Jobs::DisableBootstrapMode"} 0.038
discourse_scheduled_job_duration_seconds{job_name="Jobs::CleanUpUnusedStagedUsers"} 0.065
# HELP discourse_scheduled_job_count Total number of scheduled jobs executued
# TYPE discourse_scheduled_job_count counter
discourse_scheduled_job_count{job_name="Jobs::PendingUsersReminder"} 38
discourse_scheduled_job_count{job_name="Jobs::WhosOnlineGoingOffline"} 187
discourse_scheduled_job_count{job_name="Jobs::PresenceChannelAutoLeave"} 187
discourse_scheduled_job_count{job_name="Jobs::ProcessBadgeBacklog"} 188
discourse_scheduled_job_count{job_name="Jobs::TopicTimerEnqueuer"} 187
discourse_scheduled_job_count{job_name="Jobs::Heartbeat"} 63
discourse_scheduled_job_count{job_name="Jobs::ProcessShelvedNotifications"} 38
discourse_scheduled_job_count{job_name="Jobs::EnqueueOnceoffs"} 19
discourse_scheduled_job_count{job_name="Jobs::BookmarkReminderNotifications"} 38
discourse_scheduled_job_count{job_name="Jobs::CleanUpUploads"} 4
discourse_scheduled_job_count{job_name="Jobs::PollMailbox"} 38
discourse_scheduled_job_count{job_name="Jobs::TopRefreshToday"} 4
discourse_scheduled_job_count{job_name="Jobs::DestroyOldDeletionStubs"} 7
discourse_scheduled_job_count{job_name="Jobs::ActivationReminderEmails"} 2
discourse_scheduled_job_count{job_name="Jobs::UnsilenceUsers"} 13
discourse_scheduled_job_count{job_name="Jobs::PeriodicalUpdates"} 13
discourse_scheduled_job_count{job_name="Jobs::MigrateUploadScheme"} 19
discourse_scheduled_job_count{job_name="Jobs::AboutStats"} 6
discourse_scheduled_job_count{job_name="Jobs::CheckOutOfDateThemes"} 1
discourse_scheduled_job_count{job_name="Jobs::EnqueueDigestEmails"} 7
discourse_scheduled_job_count{job_name="Jobs::DirectoryRefreshDaily"} 3
discourse_scheduled_job_count{job_name="Jobs::DashboardStats"} 6
discourse_scheduled_job_count{job_name="Jobs::PendingReviewablesReminder"} 3
discourse_scheduled_job_count{job_name="Jobs::CreateMissingAvatars"} 3
discourse_scheduled_job_count{job_name="Jobs::UpdateAnimatedUploads"} 3
discourse_scheduled_job_count{job_name="Jobs::EnqueueSuspectUsers"} 2
discourse_scheduled_job_count{job_name="Jobs::UpdateSitemaps"} 3
discourse_scheduled_job_count{job_name="Jobs::PendingQueuedPostsReminder"} 3
discourse_scheduled_job_count{job_name="Jobs::ReindexSearch"} 1
discourse_scheduled_job_count{job_name="Jobs::DisableBootstrapMode"} 1
discourse_scheduled_job_count{job_name="Jobs::CleanUpUnusedStagedUsers"} 1
# HELP discourse_sidekiq_job_duration_seconds Total time spent in sidekiq jobs
# TYPE discourse_sidekiq_job_duration_seconds counter
discourse_sidekiq_job_duration_seconds{job_name="Jobs::RunHeartbeat"} 0.226169511675834654e0
discourse_sidekiq_job_duration_seconds{job_name="Jobs::NotifyMailingListSubscribers"} 0.8942126949783414593e1
discourse_sidekiq_job_duration_seconds{job_name="Jobs::PullHotlinkedImages"} 0.8282361692283302553e1
discourse_sidekiq_job_duration_seconds{job_name="Jobs::FeatureTopicUsers"} 0.1046757272724062204e2
discourse_sidekiq_job_duration_seconds{job_name="Jobs::BotInput"} 0.114331910074222833095e3
discourse_sidekiq_job_duration_seconds{job_name="Jobs::PostUpdateTopicTrackingState"} 0.17196641702670603947e2
discourse_sidekiq_job_duration_seconds{job_name="Jobs::CrawlTopicLink"} 0.1373126078015193336375e3
discourse_sidekiq_job_duration_seconds{job_name="Jobs::PostAlert"} 0.22092239051265641677e3
discourse_sidekiq_job_duration_seconds{job_name="Jobs::ProcessPost"} 0.30518020779779181136e3
# HELP discourse_sidekiq_job_count Total number of sidekiq jobs executed
# TYPE discourse_sidekiq_job_count counter
discourse_sidekiq_job_count{job_name="Jobs::RunHeartbeat"} 63
discourse_sidekiq_job_count{job_name="Jobs::NotifyMailingListSubscribers"} 94
discourse_sidekiq_job_count{job_name="Jobs::PullHotlinkedImages"} 100
discourse_sidekiq_job_count{job_name="Jobs::FeatureTopicUsers"} 101
discourse_sidekiq_job_count{job_name="Jobs::BotInput"} 217
discourse_sidekiq_job_count{job_name="Jobs::PostUpdateTopicTrackingState"} 101
discourse_sidekiq_job_count{job_name="Jobs::CrawlTopicLink"} 414
discourse_sidekiq_job_count{job_name="Jobs::PostAlert"} 106
discourse_sidekiq_job_count{job_name="Jobs::ProcessPost"} 106
# HELP discourse_postgres_readonly_mode Indicates whether site is in readonly mode due to PostgreSQL failover
# TYPE discourse_postgres_readonly_mode gauge
discourse_postgres_readonly_mode 0
# HELP discourse_redis_master_available Whether or not we have an active connection to the master Redis
# TYPE discourse_redis_master_available gauge
discourse_redis_master_available 1
# HELP discourse_redis_slave_available Whether or not we have an active connection a Redis slave
# TYPE discourse_redis_slave_available gauge
discourse_redis_slave_available 0
# HELP discourse_postgres_master_available Whether or not we have an active connection to the master PostgreSQL
# TYPE discourse_postgres_master_available gauge
discourse_postgres_master_available 1
# HELP discourse_postgres_replica_available Whether or not we have an active connection to the replica PostgreSQL
# TYPE discourse_postgres_replica_available gauge
# HELP discourse_active_app_reqs Number of active web requests in progress
# TYPE discourse_active_app_reqs gauge
discourse_active_app_reqs 16
# HELP discourse_queued_app_reqs Number of queued web requests
# TYPE discourse_queued_app_reqs gauge
discourse_queued_app_reqs 95
# HELP discourse_sidekiq_jobs_enqueued Number of jobs queued in the Sidekiq worker processes
# TYPE discourse_sidekiq_jobs_enqueued gauge
discourse_sidekiq_jobs_enqueued{queue="default"} 0
discourse_sidekiq_jobs_enqueued{queue="low"} 0
discourse_sidekiq_jobs_enqueued{queue="critical"} 0
discourse_sidekiq_jobs_enqueued{queue="ultra_low"} 0
# HELP discourse_sidekiq_processes Number of Sidekiq job processes
# TYPE discourse_sidekiq_processes gauge
discourse_sidekiq_processes 1
# HELP discourse_sidekiq_paused Whether or not Sidekiq is paused
# TYPE discourse_sidekiq_paused gauge
# HELP discourse_sidekiq_workers Total number of active sidekiq workers
# TYPE discourse_sidekiq_workers gauge
discourse_sidekiq_workers 5
# HELP discourse_sidekiq_jobs_stuck Number of sidekiq jobs which have been running for more than 120 minutes
# TYPE discourse_sidekiq_jobs_stuck gauge
# HELP discourse_scheduled_jobs_stuck Number of scheduled jobs which have been running for more than their expected duration
# TYPE discourse_scheduled_jobs_stuck gauge
# HELP discourse_missing_s3_uploads Number of missing uploads in S3
# TYPE discourse_missing_s3_uploads gauge
# HELP discourse_version_info Labelled with `revision` (current core commit hash), and `version` (Discourse::VERSION::STRING)
# TYPE discourse_version_info gauge
discourse_version_info{revision="e3164671698aeef94cf2e142903a30e4688e9dfe",version="2.8.0.beta6"} 1
# HELP discourse_readonly_sites Count of sites currently in readonly mode, grouped by the relevant key from Discourse::READONLY_KEYS
# TYPE discourse_readonly_sites gauge
discourse_readonly_sites{key="readonly_mode"} 0
discourse_readonly_sites{key="readonly_mode:postgres"} 0
discourse_readonly_sites{key="readonly_mode:user"} 0
discourse_readonly_sites{key="readonly_mode:postgres_force"} 0
Questions:
-
Is it DDOS or bruteforce attack on the forum or some bots traffic?
OR its genuine traffic and we need to upgrade our droplet specs, if so, what should we upgrade, by how much? -
What should we do ? right now, our site is loading very slow, and navigation from 1 topic to another, the loading transition is like 5 seconds+
-
How to find out which traffic or resource is using this much traffic? Whether if its real human traffic or bots trying to hit down our site?
-
What changes should we do in our app.yaml if we decide to upgrade our droplet specs?
-
On users profile admin page, we see unique IP of each user, means CF is getting real IPs.? but they why we get 429 too many requests error when we uncomment the ratelimited template from app.yml
-
How should we configure our CF DDOS settings?
-
Is there any way to configure our server/discourse to work in multi threaded or parallel way to support our high traffic using our same current specs droplet? If yes, how to configure multithreaded/parallel scaling in discourse to use multi core/CPUs droplets effectively and in best optimized way?
We are using cloudflare and when we turn on CF I am under attack mode, then CPU usage goes down, but we cannot annoy our new users to solve captcha each time they use our site. Right now CF bot fight mode is ON and CF security level is high.
We have tried rebuilding but after rebuild is done, it goes to normal but after few minutes CPU usage goes back to 100%
Please someone help.
