Site advise shows:
“Sidekiq is not running. Many tasks, like sending emails, are executed asynchronously by sidekiq. Please ensure at least one sidekiq process is running. Learn about Sidekiq here.”
Manually trigger jobs in sidekiq seems does not work either( I tried return heartbeat task but looks like it failed.
Any suggestion on how to bring my site back to normal?
try a rebuild via the command line and not the web interface
Thanks! I just tried a few commands:
./launcher rebuild app
-> Cannot connect to the docker daemon - verify it is running and you have access
service docker start
-> return nothing
docker info
-> fatal error: runtime: out of memory
runtime.throw(0x561377414b98, 0x16)
/usr/local/go/src/runtime/panic.go:774 +0x74 fp=0x7ffc1cb23058 sp=0x7ffc1cb23028 pc=0x561375eccf24
runtime.sysMap(0xc000000000, 0x4000000, 0x56137903e658)
/usr/local/go/src/runtime/mem_linux.go:169 +0xc7 fp=0x7ffc1cb23098 sp=0x7ffc1cb23058 pc=0x561375eb6a67
runtime.(*mheap).sysAlloc(0x5613790252c0, 0x2000, 0x0, 0x0)
/usr/local/go/src/runtime/malloc.go:701 +0x1cf fp=0x7ffc1cb23140 sp=0x7ffc1cb23098 pc=0x561375ea996f
runtime.(*mheap).grow(0x5613790252c0, 0x1, 0xffffffff)
/usr/local/go/src/runtime/mheap.go:1255 +0xa5 fp=0x7ffc1cb23190 sp=0x7ffc1cb23140 pc=0x561375ec4c05
runtime.(*mheap).allocSpanLocked(0x5613790252c0, 0x1, 0x56137903e668, 0x1)
/usr/local/go/src/runtime/mheap.go:1170 +0x268 fp=0x7ffc1cb23210 sp=0x7ffc1cb23190 pc=0x561375ec4a98
runtime.(*mheap).alloc_m(0x5613790252c0, 0x1, 0x561375ef002a, 0x0)
/usr/local/go/src/runtime/mheap.go:1022 +0xc6 fp=0x7ffc1cb23260 sp=0x7ffc1cb23210 pc=0x561375ec4336
runtime.(*mheap).alloc.func1()
/usr/local/go/src/runtime/mheap.go:1093 +0x4e fp=0x7ffc1cb23298 sp=0x7ffc1cb23260 pc=0x561375ef642e
runtime.(*mheap).alloc(0x5613790252c0, 0x1, 0x1002a, 0x40000)
/usr/local/go/src/runtime/mheap.go:1092 +0x8c fp=0x7ffc1cb232e8 sp=0x7ffc1cb23298 pc=0x561375ec461c
runtime.(*mcentral).grow(0x5613790260f8, 0x0)
/usr/local/go/src/runtime/mcentral.go:255 +0x7d fp=0x7ffc1cb23328 sp=0x7ffc1cb232e8 pc=0x561375eb648d
runtime.(*mcentral).cacheSpan(0x5613790260f8, 0x7fb47f218000)
/usr/local/go/src/runtime/mcentral.go:106 +0x300 fp=0x7ffc1cb23388 sp=0x7ffc1cb23328 pc=0x561375eb5fb0
runtime.(*mcache).refill(0x7fb47f218008, 0x2a)
/usr/local/go/src/runtime/mcache.go:138 +0x87 fp=0x7ffc1cb233a8 sp=0x7ffc1cb23388 pc=0x561375eb5a47
runtime.(*mcache).nextFree(0x7fb47f218008, 0x56137903e62a, 0x7fb47f218008, 0x8, 0xfffffffffffffff8)
/usr/local/go/src/runtime/malloc.go:854 +0x89 fp=0x7ffc1cb233e0 sp=0x7ffc1cb233a8 pc=0x561375eaa199
runtime.mallocgc(0x180, 0x561378084760, 0x1, 0x56137903e6c0)
/usr/local/go/src/runtime/malloc.go:1022 +0x7a5 fp=0x7ffc1cb23480 sp=0x7ffc1cb233e0 pc=0x561375eaaae5
runtime.newobject(0x561378084760, 0x4000)
/usr/local/go/src/runtime/malloc.go:1151 +0x3a fp=0x7ffc1cb234b0 sp=0x7ffc1cb23480 pc=0x561375eaaeea
runtime.malg(0x33eb6a00008000, 0x561379027968)
/usr/local/go/src/runtime/proc.go:3232 +0x33 fp=0x7ffc1cb234f0 sp=0x7ffc1cb234b0 pc=0x561375ed63f3
runtime.mpreinit(...)
/usr/local/go/src/runtime/os_linux.go:324
runtime.mcommoninit(0x56137901f580)
/usr/local/go/src/runtime/proc.go:626 +0xc6 fp=0x7ffc1cb23528 sp=0x7ffc1cb234f0 pc=0x561375ecfd26
runtime.schedinit()
/usr/local/go/src/runtime/proc.go:543 +0x76 fp=0x7ffc1cb23580 sp=0x7ffc1cb23528 pc=0x561375ecf986
runtime.rt0_go(0x7ffc1cb23688, 0x2, 0x7ffc1cb23688, 0x0, 0x7fb47e84ab97, 0x2, 0x7ffc1cb23688, 0x200008000, 0x561375ef8780, 0x0, ...)
/usr/local/go/src/runtime/asm_amd64.s:214 +0x129 fp=0x7ffc1cb23588 sp=0x7ffc1cb23580 pc=0x561375ef88b9
What information I need or what can I do to further investigate this?
try
./launcher destroy app
./launcher start app
both commands give me Cannot connect to the docker daemon - verify it is running and you have access
then use sudo to run the docker
sudo service docker start
this returns nothing and rerun ./launcher start app still shows same " Cannot connect to the docker daemon" error.
BTW, free -h result, I experienced one error update 4 days ago and from that time disk and cpu shows spike metrics.
+-------+-------+------+------+--------+------------+-----------+
| | total | used | free | shared | buff/cache | available |
+-------+-------+------+------+--------+------------+-----------+
| Mem: | 1.9G | 1.6G | 78M | 194M | 246M | 30M |
+-------+-------+------+------+--------+------------+-----------+
| Swap: | 2.0G | 2.0G | 0B | | | |
+-------+-------+------+------+--------+------------+-----------+
will I lost all data if I just restart my instance?
Not sure anyone can 100% answer this question, but the only thing todo is a reboot.
For anyone who has the same issue, an instance reboot works for me and RAM/Swap back to normal:
+-------+-------+------+------+--------+------------+-----------+
| | total | used | free | shared | buff/cache | available |
+-------+-------+------+------+--------+------------+-----------+
| Mem: | 1.9G | 985M | 76M | 40M | 931M | 785M |
+-------+-------+------+------+--------+------------+-----------+
| Swap: | 2.0G | 6.8M | 2.0G | | | |
+-------+-------+------+------+--------+------------+-----------+
then I just rerun ./launcher rebuild app and everything works again, sidekiq starts to work.
Thank you so much for all your help! 
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.