How to stop a kernel update from forcing endless reboots using dmesg and journalctl -

When a kernel update turns your machine into a reboot loop

Kernel upgrades are usually painless, but occasionally a new module or a subtle change in the initramfs can trigger a panic that forces the system to reboot immediately. The result is an endless cycle of boot → panic → reboot that can lock you out of the box. The good news is that the kernel’s own logs are usually the first place to look, and systemd’s journal gives you a convenient, persistent view of what happened before the crash.

Why a kernel update can force a reboot

A kernel panic is the kernel’s way of saying “I can’t recover from this state.” When the panic handler runs, it usually calls reboot() to give the machine a clean slate. On most distributions, the default systemd configuration (systemd-reboot.service) will trigger a hard reboot as soon as the panic occurs. If the panic happens during early boot—say, while loading a new driver—the machine will never reach a point where you can log in to fix the issue.

Common culprits in 2026 include:

A new driver that misbehaves on older hardware.
A change in the initramfs that pulls in a module with a faulty firmware blob.
A bug in the kernel’s handling of a particular PCI device that surfaces only after a recent patch.

Because the kernel is already in a compromised state, you can’t rely on the usual apt, dnf, or zypper tools to roll back the kernel until you’ve stopped the reboot loop.

Diagnosing with `dmesg`

dmesg shows the kernel ring buffer, which contains messages from the current boot. If you can get the machine to a state where you can run a shell—perhaps by booting into a rescue mode or a previous kernel—you can inspect the last few lines that caused the panic.

# Show the last 200 lines, with timestamps
dmesg -T | tail -n 200

Typical panic output looks like:

[    1.234567] BUG: unable to handle kernel NULL pointer dereference at ...
[    1.234567] IP: ...  [<ffffffff81012345>] mymodule+0x123/0x456
[    1.234567] PGD 0x0 P4D 0x0 PUD 0x0 PMD 0x0
[    1.234567] Oops: 0000 [#1] SMP
[    1.234567] Modules linked in: mymodule
[    1.234567] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G           5.15.0-rc7+ #1

Notice the mymodule name; that’s the module that triggered the panic. If the message mentions a hardware device (e.g., i915, nvme, xhci_hcd), you know which driver to investigate.

Diagnosing with `journalctl`

Systemd’s journal keeps a persistent log of all boot sessions. Even if the machine reboots before you can log in, the previous boot’s logs are still available.

# Show the last boot (the one that crashed)
journalctl -b -1 | grep -i panic -n

You can also filter by the kernel unit:

journalctl -b -1 -k

The -k flag shows only kernel messages, which is useful when you’re hunting for a panic. Look for the same BUG or Oops lines that appear in dmesg. The journal will also contain the exact kernel version that crashed, which is handy when you need to compare against a known‑good release.

Stopping the loop temporarily

If you can’t get to a shell, you can still prevent the kernel from rebooting automatically by editing the systemd-reboot.service unit. This is a quick, non‑persistent fix that lets you boot into the problematic kernel once, investigate, and then reboot normally.

sudo systemctl edit systemd-reboot.service

Add the following snippet:

[Service]
ExecStart=
ExecStart=/usr/bin/false

This overrides the default ExecStart that triggers a reboot, replacing it with a harmless command. Reload systemd and reboot:

sudo systemctl daemon-reload
sudo reboot

Now the machine will stop at the kernel panic screen, allowing you to attach a serial console or use a rescue image to inspect the logs.

Trade‑off: Disabling automatic reboot hides the fact that the kernel is in an unrecoverable state. Use this only for debugging; re‑enable it once you’ve fixed the issue.

Permanently disabling the reboot for a specific kernel

If you know that a particular kernel version is problematic, you can pin the system to an older, stable kernel and prevent the new one from being used automatically.

List installed kernels:
```
dpkg --list | grep linux-image
```
Set the default kernel (Debian/Ubuntu style):
```
sudo dpkg-reconfigure linux-image-5.15.0-28-generic
```
For RPM‑based systems, edit /etc/default/grub and set GRUB_DEFAULT to the desired entry, then run sudo grub2-mkconfig -o /boot/grub2/grub.cfg.
Mask the new kernel’s initramfs (optional):
```
sudo systemctl mask initrd-5.15.0-29-generic.service
```
This prevents the initramfs from being generated for the offending kernel, which can be useful if the initramfs itself is the source of the panic.

Fixing the underlying issue

Once you’ve stopped the reboot loop and have access to the logs, the next step is to address the root cause.

Kernel module mismatch: If a module is missing or incompatible, reinstall the kernel headers and rebuild the module.
```
sudo apt-get install --reinstall linux-headers-$(uname -r)
sudo dkms autoinstall
```
Firmware bug: Some drivers rely on firmware blobs that may have been updated. Re‑install the firmware package.
```
sudo apt-get install --reinstall linux-firmware
```
Hardware incompatibility: If the panic is tied to a specific device, check the hardware’s vendor site or the kernel’s own documentation. Sometimes a simple BIOS/UEFI update resolves the issue.
Kernel.org hosts the official source and changelogs: https://www.kernel.org/

Using `systemd`’s crash‑dump feature

Systemd can capture a crash dump via kdump or systemd-coredump. Enabling this early in the boot process gives you a post‑mortem that survives a reboot.

sudo systemctl enable kdump.service
sudo systemctl start kdump.service

After a crash, the dump will be in /var/crash/. Inspect it with coredumpctl:

coredumpctl list
coredumpctl gdb <PID>

This is especially useful when the panic occurs before the initramfs is fully loaded, as the dump contains the kernel’s memory state at the moment of failure.

Security trade‑offs

Disabling automatic reboot can expose the system to unpatched vulnerabilities if you keep a buggy kernel running. If you’re in a production environment, consider:

Using a temporary rescue kernel: Boot into a known‑good kernel until you can patch or replace the faulty one.
Restricting remote access while the buggy kernel is active, so you’re not exposed to potential exploits that rely on the kernel panic state.

Remember, the goal is to get a stable boot, not to keep a broken kernel running for long. Once you’ve identified the culprit

linux open-source

When a kernel update turns your machine into a reboot loop

Why a kernel update can force a reboot

Diagnosing with dmesg

Diagnosing with journalctl