I ran into this issue last night, when my Ubuntu 20.04 LTS automatically upgraded itself, it installed a new kernel and I lost control of the system, it would just crash a few minutes after booting. I tried it again today with a fresh Discourse install and as soon as I upgraded the system it started crashing again. Just a note for folks, don’t update your linux kernels just yet, this is a known bug - see this for more details. Bug #1977919 “Docker container creation causes kernel oops on li...” : Bugs :... Running the attached script on the latest AWS AMI for Ubuntu 20.04, I get a kernel panic and hard reset of the node. [ 12.314552] VFS: Close: file count is 0 [ 12.351090] ------------[ cut here ]------------ [ 12.351093] kernel BUG at...

The question being is there a way to start the system without having it start Discourse/docker? Running on AWS Lightsail. The only other option is to rebuild the whole system again which is PITA right now given the backup/restore issues I’m facing. EDIT: This is what I found, hit or miss depending on how fast it comes up. while true; do ssh "sudo systemctl disable docker.service; sudo systemctl disable containerd.service" done

I had this happen on two EC2 instances as well. They went down at 5AM EDT for a reboot and never came back up.

Per the link this affects people running canonical “cloud kernels” in ubuntu machines. They removed a patch that affects OverlayFS. While Canonical rolls a fix people can try a different kernel version or using Debian / other distro as a workaround.

I was able to interrupt the cycle using a quick SSH about 15 seconds after it starts to disable the docker/container services. Downgraded the kernel to 5.4 and it seems to be working [תמונה] How to downgrade the kernel on Ubuntu 20.04 to the 5.4 (LTS) version Documentation Tutorials server cloud No matter what your reasons are, you may be in a situation where you need to downgrade your kernel to a specific version. In this guide, I will show you how to downgrade a recent Ubuntu Pro 20.04 AMI’s kernel to the original LTS version (5.4) ... Reading time: 1 mins 🕑 Likes: 2 ❤

[תמונה] RBoy: I was able to interrupt the cycle using a quick SSH about 15 seconds after it starts to disable the docker/container services. Downgraded the kernel to 5.4 and it seems to be working How to downgrade the kernel on Ubuntu 20.04 to the 5.4 (LTS) version - Tutorials - Ubuntu Community Hub Yes, as I just posted in your other thread on restore troubles, that was essentially what I did as well when this bug crashed my server. Well, I booted the old kernel; didn’t have to disable docker or containers. And the current kernel is safe again. Here is a link to what I said in your other thread. In a bit I’ll try to write up my permanent solution to keep this from happening again. Nasty kernel bug, that was!

You simply can revert to the previous kernel and the machine is restored. Or update to the current, fixed kernel, which came out on Thursday.

I have written up a tutorial on how to avoid kernel oops! issues like this that crash your server or keep it from coming back up. I put the tutorial on my Discourse site, since that seemed convenient to me. My site has nothing to do with tech, though. So I unlisted the topic but published it to HTML. Enjoy. https://discourse.bluebottlefly.com/pub/hardening-your-server @RBoy, maybe you in particular will find this useful. /dr

עדכון ליבת Ubuntu 20.04 עם Docker גורם לקריסה ב-EC2 וב-Lightsail

תמיכה התקנה

RBoy (RBoy) 9 ביוני,‏ 2022,‏ 5:28pm 6

I was able to interrupt the cycle using a quick SSH about 15 seconds after it starts to disable the docker/container services. Downgraded the kernel to 5.4 and it seems to be working

נושא		תגובות	צפיות
Kernel 1028 (linux-aws. linux-gcp and linux-azure) breaks docker and therefore Discourse installs Product Support	2	652	10 ביוני,‏ 2022
Kernel version upgrade required when manually updating discourse Self-hosting	35	4766	8 ביוני,‏ 2023
Ubuntu vm upgrade from 22 to 24 holding discourse docker Product Support	4	387	8 בנובמבר,‏ 2024
Bootstrap app causes system crash and reboot Self-hosting	5	1817	1 במאי,‏ 2016
Ubuntu updates intefere with Docker and AUFS? Self-hosting	17	14723	2 במרץ,‏ 2016

עדכון ליבת Ubuntu 20.04 עם Docker גורם לקריסה ב-EC2 וב-Lightsail

נושאים קשורים