Why Your systemd Service Stays “activating” After a Kernel Upgrade – A Step‑by‑Step Fix​

Why a service can stay “activating” after a kernel upgrade

When you bump the kernel, systemd does a hard reset of almost everything.
If a unit you depend on never leaves the activating state, the rest of the boot can stall or the service simply never runs.
The usual suspect is a dependency on a kernel‑provided resource that is missing or delayed after the upgrade.

Below is a practical checklist that shows how to diagnose the issue, why it happens, and how to fix it without compromising security or stability.


1. Spot the symptom early

systemctl status my‑service.service

Typical output:

 my-service.service
   Loaded: loaded (/etc/systemd/system/my-service.service; enabled)
   Active: activating (start) since Mon 20260904 10:12:34 UTC; 3s ago

If the state never changes, run:

journalctl -u my-service.service -b

Look for lines such as:

my-service.service: Failed to start: Device or resource busy
my-service.service: Timeout waiting for device /dev/sdb1

These clues point to a missing kernel module, a device that never appears, or a udev rule that didn’t fire.


2. Common causes after a kernel upgrade

CauseWhy it happensTypical symptom
Missing kernel moduleThe new kernel no longer autoloads a module that the service expects.modprobe fails, or the device node never appears.
Device name changeudev rules or the kernel’s naming scheme changed (e.g., from sda to nvme0n1).Service waits for /dev/sda1 that never exists.
Udev settle delaysystemd-udev-settle.service runs longer because the kernel emits many events.Service has After=systemd-udev-settle.service but the settle service hangs.
Initramfs missing modulesThe initramfs used by the new kernel lacks a module needed early in boot.Services that need the module fail during the initramfs stage.

3. Step‑by‑step diagnosis

  1. Check the unit’s dependencies

    systemctl show my-service.service --property=After,Requires
    

    If you see systemd-udev-settle.service or systemd-udevd.service, the unit waits for udev.

  2. Verify the kernel modules

    lsmod | grep <module_name>
    

    If the module is missing, try loading it manually:

    sudo modprobe <module_name>
    

    If it loads, add it to /etc/modules-load.d/<name>.conf to persist across reboots.

  3. Inspect udev events

    udevadm monitor --environment --property
    

    Trigger the device manually:

    sudo udevadm trigger /dev/<device>
    

    Then check if the service starts.

  4. Look at the udev settle timeout

    systemctl show systemd-udev-settle.service --property=TimeoutStartUSec
    

    The default is 30 s. If it’s too short for your hardware, increase it:

    # /etc/systemd/system/systemd-udev-settle.service.d/override.conf
    [Service]
    TimeoutStartSec=60
    

    Reload the unit:

    systemctl daemon-reload
    
  5. Rebuild the initramfs if needed

    On Debian/Ubuntu:

    sudo update-initramfs -u -k $(uname -r)
    

    On RHEL/CentOS:

    sudo dracut -v -f
    

    This ensures that modules required early in boot are present.


4. Fixing the unit file

Once you know the missing piece, adjust the unit file.

4.1 Add a module load step

[Unit]
Description=My Service
After=systemd-udev-settle.service
Requires=systemd-udev-settle.service

[Service]
ExecStartPre=/sbin/modprobe <module_name>
ExecStart=/usr/bin/my-service

4.2 Force a udev settle with a timeout

ExecStartPre=/usr/bin/udevadm settle --timeout=30

This guarantees that all pending udev events finish before the service starts.

4.3 Use PartOf= instead of After= when appropriate

If the service should restart when the kernel module reloads, link it with:

PartOf=systemd-modules-load.service

This keeps the unit in sync with the kernel state.


5. Security considerations

  • Least privilege – Run the service as a dedicated user:

    [Service]
    User=myuser
    Group=mygroup
    
  • Capability bounding – Drop unnecessary capabilities:

    [Service]
    CapabilityBoundingSet=CAP_NET_BIND_SERVICE
    
  • Sandboxing – If the service doesn’t need to touch the rest of the system, use PrivateTmp, ProtectSystem=full, etc.


#TAGS: systemd, kernel, troubleshooting, services


See also