I like many existing users I have been experiencing a combination of these issues:
Low memory errors
ImageMagicK message in admin panel.
White pages / instability
This post / topic is a dump of available information before I restart the instance - so it might help track down the problem.
###System
###History ./launcher restart app to resolve my then availability issues.
Current version: Discourse 1.2.0.beta6 - https://github.com/discourse/discourse version 8689c85b54efccbccfa216c800522e5ba1a79dd1
###Possible related reportsmay not be the the same issue!
Since updating to 1.2.0.beta6 my site has crashed twice and needed to be rebuilt twice to fix it in a 3 day period.
Tested from multiple systems and users using Mac (Safari), Windows 8.1 (FireFox, Chrome, IE), Android Lollipop (Chrome, FireFox), iOS 8 (Safari)
Rebuilding seems to fix it, but it crashes again after about 24-30 hours. I am running default discourse, no plugins, the specs of my server are 4 times the recommended settings, no massive traffic spike or issues on the server, using th…
https://meta.discourse.org/t/memory-usage-and-blank-pages/24511
###Digital Ocean graphs
###Pingdom uptime graphs
Still collecting information and will continue to post in this thread.
4 Likes
I believe ruby / some processed restarted not long after I logged into the shell and the site became “more available” displaying pages:
%Cpu(s): 33.4 us, 9.0 sy, 0.0 ni, 38.9 id, 18.6 wa, 0.2 hi, 0.0 si, 0.0 st
KiB Mem: 2049988 total, 1983064 used, 66924 free, 136 buffers
KiB Swap: 2097148 total, 2082940 used, 14208 free. 349796 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20572 message+ 20 0 413312 277384 257608 R 68.7 13.5 0:17.45 postmaster
17653 1000 20 0 1126156 178964 2996 S 9.3 8.7 0:15.72 ruby
36 root 20 0 0 0 0 S 3.3 0.0 80:30.07 kswapd0
28638 1000 20 0 1685276 305940 3064 S 1.0 14.9 40:48.71 ruby
19441 root 20 0 24956 648 364 R 0.7 0.0 0:01.91 top
28547 landsca+ 20 0 435592 55272 688 S 0.3 2.7 20:40.51 redis-server
28569 message+ 20 0 386336 107708 107172 S 0.3 5.3 0:46.53 postmaster
28572 message+ 20 0 103908 800 248 S 0.3 0.0 0:46.54 postmaster
1 root 20 0 33492 272 0 S 0.0 0.0 1:54.08 init
Note that postmaster is using a lot of CPU - this normally only happens when starting for me.
My current top looks like this:
Tasks: 116 total, 2 running, 114 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.5 us, 0.3 sy, 0.0 ni, 99.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 2049988 total, 1980548 used, 69440 free, 1208 bufferss-server
KiB Swap: 2097148 total, 2074824 used, 22324 free. 307812 cached Memter
36 root 20 0 0 0 0 S 3.3 0.0 80:30.26 kswapd0
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7 root 20 0 0 0 0 S 0.3 0.0 23:09.09 rcu_sched
17653 1000 20 0 1132308 182752 3936 S 0.3 8.9 0:22.23 ruby
18099 root 20 0 105632 260 160 S 0.3 0.0 0:00.90 sshd
21762 root 20 0 24952 1592 1096 R 0.3 0.1 0:00.02 top
28547 landsca+ 20 0 435592 56044 712 S 0.3 2.7 20:44.53 redis-server
28656 1000 20 0 1664796 288848 2432 S 0.3 14.1 40:53.90 ruby
1 root 20 0 33492 264 0 S 0.0 0.0 1:54.10 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.70 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 1:36.78 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
8 root 20 0 0 0 0 R 0.0 0.0 11:06.96 rcuos/0
Current login system displays high swap / memory usage
System information as of Mon Feb 9 15:04:46 EST 2015
System load: 0.17 Processes: 119
Usage of /: 63.7% of 39.25GB Users logged in: 0
Memory usage: 81% IP address for eth0:
Swap usage: 95% IP address for docker0:
8 hours ago the following noteworthy items appeared in the logs
##Log
###Error 1#Info
####Backtrace ' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/core_ext/kernel/agnostics.rb:7:in ’resize' /var/www/discourse/lib/letter_avatar.rb:44:in generate’show_letter' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/implicit_render.rb:4:in send_action’process_action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/rendering.rb:10:in process_action’block in process_action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:113:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:149:in block in halting_and_conditional’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:149:in block in halting_and_conditional’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:229:in block in halting’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:149:in block in halting_and_conditional’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:86:in run_callbacks’process_action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/rescue.rb:29:in process_action’block in process_action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/notifications.rb:159:in block in instrument’instrument' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/notifications.rb:159:in instrument’process_action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/params_wrapper.rb:250:in process_action’process_action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/abstract_controller/base.rb:136:in process’process' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-mini-profiler-0.9.2/lib/mini_profiler/profiling_methods.rb:79:in block in profile_method’dispatch' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/rack_delegation.rb:13:in dispatch’block in action' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:82:in call’dispatch' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:50:in call’block in call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/journey/router.rb:59:in each’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:678:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in call!’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in call!’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in call!’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in call!’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in call!’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in call!’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/builder.rb:59:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/head.rb:11:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/params_parser.rb:27:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.6/lib/message_bus/rack/middleware.rb:55:in call’context' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/query_cache.rb:36:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/callbacks.rb:29:in block in call’run_callbacks' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/callbacks.rb:27:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/debug_exceptions.rb:17:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-0.1.6/lib/logster/middleware/reporter.rb:23:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/request_id.rb:21:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/runtime.rb:17:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-mini-profiler-0.9.2/lib/mini_profiler/profiler.rb:193:in call’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/application.rb:144:in call’public_send' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/railtie.rb:194:in method_missing’call' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/urlmap.rb:65:in block in call’each' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in call’process_client' /var/www/discourse/lib/middleware/unicorn_oobgc.rb:95:in process_client’worker_loop' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:525:in spawn_missing_workers’start' /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/bin/unicorn:126:in <top (required)>’load' /var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in ’
####Env
HTTP_HOST: example.com
REQUEST_URI: /letter_avatar/isaac/40/2.png
REQUEST_METHOD: GET
HTTP_USER_AGENT: Mozilla/5.0 (iPad; CPU OS 8_1_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B440 Safari/600.1.4
HTTP_ACCEPT: */*
HTTP_REFERER: http://example.com/t/topic/36090/5
###Error 2#Info
Errno::ENOMEM (Cannot allocate memory - convert)
####Backtrace
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``'
/var/www/discourse/app/models/upload.rb:134:in `fix_image_orientation'
/var/www/discourse/app/models/upload.rb:106:in `resize_image'
/var/www/discourse/app/models/upload.rb:77:in `create_for'
/var/www/discourse/app/controllers/uploads_controller.rb:8:in `create'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/abstract_controller/base.rb:189:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:149:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:229:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:229:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:166:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:86:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:86:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/rescue.rb:29:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/notifications.rb:159:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/notifications.rb:159:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/abstract_controller/base.rb:136:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.1.8/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-mini-profiler-0.9.2/lib/mini_profiler/profiling_methods.rb:79:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal.rb:196:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_controller/metal.rb:232:in `block in action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:82:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/journey/router.rb:73:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/journey/router.rb:59:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/journey/router.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/routing/route_set.rb:678:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/omniauth-1.2.2/lib/omniauth/builder.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/conditionalget.rb:35:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/head.rb:11:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:119:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/flash.rb:254:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/message_bus-1.0.6/lib/message_bus/rack/middleware.rb:55:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:225:in `context'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/session/abstract/id.rb:220:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/query_cache.rb:36:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.8/lib/active_support/callbacks.rb:82:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-0.1.6/lib/logster/middleware/reporter.rb:23:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/rack/logger.rb:22:in `call'
/var/www/discourse/config/initializers/quiet_logger.rb:10:in `call_with_quiet_assets'
/var/www/discourse/config/initializers/silence_logger.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.1.8/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/runtime.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/sendfile.rb:112:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-mini-profiler-0.9.2/lib/mini_profiler/profiler.rb:193:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/engine.rb:514:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/application.rb:144:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/railtie.rb:194:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.8/lib/rails/railtie.rb:194:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/builder.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/urlmap.rb:65:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/urlmap.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:576:in `process_client'
/var/www/discourse/lib/middleware/unicorn_oobgc.rb:95:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:670:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:525:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:140:in `start'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.3/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `<main>'
####Env
REQUEST_URI: /uploads.json?authenticity_token=emx0%2B3NI6Vl2QP%2BpO2TcGnBZxxxxxxxxxxxxxx%3D
REQUEST_METHOD: POST
HTTP_USER_AGENT: Mozilla/5.0 (Windows NT 6.1; rv:35.0) Gecko/20100101 Firefox/35.0
HTTP_ACCEPT: application/json, text/javascript, */*; q=0.01
HTTP_REFERER: http://example.com/t/topic/66341
username: example
params:
authenticity_token: emx0+3xxxxl2QP+pO2TcGnBZxxxxxxxxxxxxxx
files: #<ActionDispatch::Http::UploadedFile:0x007fea29205010>
riking
February 9, 2015, 9:01pm
6
Okay, this is multiple ENOMEM reports in a day. Something with our memory usage has gone up.
@riking Are there any specific commands you would like me to run / logs to check at this time before I restart?
From 36 minutes ago - ~20:26
###Error 3#Info
Failed to log exception MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error. -1025848994072858232
Reason: Redis::CommandError EXECABORT Transaction discarded because of previous errors.
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis/pipeline.rb:79:in `finish'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis/client.rb:146:in `block in call_pipeline'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis/client.rb:273:in `with_reconnect'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis/client.rb:144:in `call_pipeline'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis.rb:2147:in `block in multi'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis.rb:37:in `block in synchronize'
/usr/local/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis.rb:37:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/redis-3.2.0/lib/redis.rb:2139:in `multi'
/var/www/discourse/lib/discourse_redis.rb:40:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-0.1.6/lib/logster/redis_store.rb:17:in `save'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-0.1.6/lib/logster/base_store.rb:70:in `report'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-0.1.6/lib/logster/logger.rb:47:in `add'
/usr/local/lib/ruby/2.0.0/logger.rb:454:in `error'
/var/www/discourse/config/initializers/sidekiq.rb:60:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.3.1/lib/sidekiq/exception_handler.rb:20:in `block in handle_exception'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.3.1/lib/sidekiq/exception_handler.rb:18:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.3.1/lib/sidekiq/exception_handler.rb:18:in `handle_exception'
/var/www/discourse/lib/discourse.rb:23:in `handle_exception'
/var/www/discourse/config/initializers/sidekiq.rb:36:in `rescue in block (2 levels) in <top (required)>'
/var/www/discourse/config/initializers/sidekiq.rb:32:in `block (2 levels) in <top (required)>'
####Backtrace
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/logster-0.1.6/lib/logster/logger.rb:47:in `add'
/usr/local/lib/ruby/2.0.0/logger.rb:463:in `fatal'
/var/www/discourse/config/initializers/sidekiq.rb:62:in `rescue in call'
/var/www/discourse/config/initializers/sidekiq.rb:65:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.3.1/lib/sidekiq/exception_handler.rb:20:in `block in handle_exception'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.3.1/lib/sidekiq/exception_handler.rb:18:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.3.1/lib/sidekiq/exception_handler.rb:18:in `handle_exception'
/var/www/discourse/lib/discourse.rb:23:in `handle_exception'
/var/www/discourse/config/initializers/sidekiq.rb:36:in `rescue in block (2 levels) in <top (required)>'
/var/www/discourse/config/initializers/sidekiq.rb:32:in `block (2 levels) in <top (required)>'
####Env
current_db: default
message: While ticking scheduling manager
sam
February 9, 2015, 9:08pm
11
confirmed, will find the leak and plug it.
4 Likes
riking
February 9, 2015, 9:08pm
12
Yep, memory errors. @sam is investigating.
Perhaps we need a memory bench, to prevent this in the future?
1 Like
@sam Would you like any further information at this time? - It’s currently occurring for me.
sam
February 9, 2015, 9:09pm
14
This is a leak, we would need to run for multiple days to catch it.
1 Like
Top processes using memory.
root@forum:~# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -10
15.1 0.8 1685276 28638 unicorn worker[0] -E production -c config/unicorn.conf.rb
14.7 0.8 1670916 28648 unicorn worker[1] -E production -c config/unicorn.conf.rb
14.3 0.8 1664796 28656 unicorn worker[2] -E production -c config/unicorn.conf.rb
11.6 0.0 386440 28568 postgres: checkpointer process
10.5 0.3 420496 16637 postgres: discourse discourse [local] idle
8.9 0.9 1148692 17653 sidekiq 3.3.1 discourse [0 of 5 busy]
7.7 0.3 406816 20262 postgres: discourse discourse [local] idle
7.6 0.3 406768 20340 postgres: discourse discourse [local] idle
7.2 0.3 409860 20606 postgres: discourse discourse [local] idle
5.3 0.2 418868 28573 unicorn master -E production -c config/unicorn.conf.rb
Seems this looks like it has been confirmed as a bug - I’ll change the category.
Feel free to change it back.
Possible earlier reports of issues:
I’m not sure what’s changed, but after upgrading to 1.2.0.beta6 over the weekend, I’ve logged into the admin of my site today and seen this:
The server is configured to create thumbnails of large images, but ImageMagick is not installed. Install ImageMagick using your favorite package manager or download the latest release.
I’ve never seen this before, and I’m surprised to see it at all, as I figured that things like this was handed by the docker image? Can you help? Thanks.
After updating my forum to 1.2.0.beta6 I got this message in the Dashboard:
Some problems have been found with your installation of Discourse:
The server is configured to create thumbnails of large images, but ImageMagick is not installed. Install ImageMagick using your favorite package manager or download the latest release.
Am I supposed to install ImageMagick manually? I’m running the Docker installation on a Digital Ocean droplet.
This happens quite frequently. Only on random occasions do I get to access the admin users page. Below are the screenshots of the two console errors.
[image]
[image]
I just upgrade my discourse to the newest version (beta 6?), and it said fail to connect to the website (before upgrade, it is all ok). Then I type my site url and it shows all blank, I don’t know what’s wrong with it. So any one can help me? I didn’t even backup my website
Since updating to 1.2.0.beta6 my site has crashed twice and needed to be rebuilt twice to fix it in a 3 day period.
Tested from multiple systems and users using Mac (Safari), Windows 8.1 (FireFox, Chrome, IE), Android Lollipop (Chrome, FireFox), iOS 8 (Safari)
Rebuilding seems to fix it, but it crashes again after about 24-30 hours. I am running default discourse, no plugins, the specs of my server are 4 times the recommended settings, no massive traffic spike or issues on the server, using th…
2 Likes
During / after restarting memory usage from top processes.
root@forum:~# cd /var/discourse/
root@forum:/var/discourse#
root@forum:/var/discourse# ./launcher restart app
WARNING: No swap limit support
a33abd1122692e1672bd0892f13e28d792146dd2a519c8b7a07cc0012195ab74
cid found, ensuring container is started
a33abd1122692e1672bd0892f13e28d792146dd2a519c8b7a07cc0012195ab74
root@forum:/var/discourse# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -10
17.8 104 394636 25707 /usr/bin/redis-server *:6379
2.3 88.8 142816 25738 ruby /var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn -E production -c config/unicorn.conf.rb
1.1 1.8 386204 25710 /usr/lib/postgresql/9.3/bin/postmaster -D /etc/postgresql/9.3/main
0.6 0.0 692804 640 /usr/bin/docker -d
0.3 0.2 188332 25650 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.17.0.36 -container-port 80
0.2 1.0 20036 25745 sleep 1
0.2 0.7 121152 25706 nginx: master process /usr/sbin/nginx
0.2 0.2 188332 25641 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 2222 -container-ip 172.17.0.36 -container-port 22
0.1 18.5 29748 25705 /bin/bash config/unicorn_launcher -E production -c config/unicorn.conf.rb
0.1 0.4 61364 25712 /usr/sbin/sshd -D -e
root@forum:/var/discourse# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -10
19.1 19.1 423304 25707 /usr/bin/redis-server *:6379
11.9 17.9 497800 25796 sidekiq 3.3.1 discourse [0 of 5 busy]
10.7 21.2 457864 25806 unicorn worker[0] -E production -c config/unicorn.conf.rb
10.3 62.8 439344 25738 unicorn master -E production -c config/unicorn.conf.rb
10.3 12.3 457864 25832 unicorn worker[1] -E production -c config/unicorn.conf.rb
10.2 12.3 455728 25842 unicorn worker[2] -E production -c config/unicorn.conf.rb
9.4 57.0 398572 25810 postgres: discourse discourse [local] SELECT
7.0 16.7 405736 25848 postgres: discourse discourse [local] idle
6.0 8.4 396424 25841 postgres: discourse discourse [local] idle
4.2 16.0 397380 25818 postgres: discourse discourse [local] idle
root@forum:/var/discourse# ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -10
19.1 11.9 423304 25707 /usr/bin/redis-server *:6379
13.4 61.6 400348 25810 postgres: discourse discourse [local] SELECT
12.0 7.4 501896 25796 sidekiq 3.3.1 discourse [0 of 5 busy]
10.7 12.3 457864 25806 unicorn worker[0] -E production -c config/unicorn.conf.rb
10.3 8.2 457892 25842 unicorn worker[2] -E production -c config/unicorn.conf.rb
10.2 6.6 457864 25832 unicorn worker[1] -E production -c config/unicorn.conf.rb
10.2 38.0 439344 25738 unicorn master -E production -c config/unicorn.conf.rb
8.0 6.7 407784 25848 postgres: discourse discourse [local] idle
6.5 2.9 401448 25841 postgres: discourse discourse [local] idle
4.8 2.1 397328 25829 postgres: discourse discourse [local] idle
root@forum:/var/discourse#
riking
February 9, 2015, 9:46pm
19
Hm, it seems redis is taking up a higher percentage of real memory?
Perhaps Redis is getting extra keys gunking it up? That don’t expire properly.
What’s the general theme of the output of this command?
./launcher enter app
redis-cli INFO
redis-cli KEYS '*' | awk '{print $1}' | sort
redis-cli KEYS '*' | awk '{print $1}' | wc -l
Obviously I have restarted the instance now so not sure how useful that info will be.
No longer at console. Will run when can, atleast 4 hours time.
