Troubleshooting Btrfs Snapshot Overfill with systemd Timers and Log Rotation -

Introduction to Btrfs Snapshots

I’ve worked with Btrfs for years, and its snapshot feature is one of my favorites. It allows you to create a read-only copy of a subvolume at a particular point in time, which is super useful for backups, testing, and rolling back changes. However, managing these snapshots can become a real pain, especially when dealing with a large number of them. In this article, I’ll show you how to troubleshoot Btrfs snapshot overfill issues using systemd timers and log rotation.

Understanding Btrfs Snapshots

Before we dive into troubleshooting, let’s cover the basics of Btrfs snapshots. A snapshot is a read-only copy of a subvolume, which can be created using the btrfs snapshot command. For example:

btrfs subvolume snapshot /mnt/btrfs/root /mnt/btrfs/snapshots/20260601

This command creates a snapshot of the /mnt/btrfs/root subvolume and stores it in the /mnt/btrfs/snapshots/20260601 directory. Don’t bother with trying to create snapshots manually, though - it’s much easier to automate the process.

Managing Snapshots with Systemd Timers

To automate the creation and removal of snapshots, we can use systemd timers. I usually start with creating a timer file, like btrfs-snapshot.timer, in the /etc/systemd/system directory:

[Unit]
Description=Btrfs Snapshot Timer

[Timer]
OnUnitInactiveSec=1d
AccuracySec=1m
Unit=btrfs-snapshot.service

[Install]
WantedBy=timers.target

This timer will trigger the btrfs-snapshot.service unit every day, with a 1-minute accuracy. The real trick is to make sure the timer is enabled and running - you can check the status with systemctl status btrfs-snapshot.timer.

Creating the Snapshot Service

The btrfs-snapshot.service unit will contain the script that creates a new snapshot and removes old ones. I like to create a file called btrfs-snapshot.service in the /etc/systemd/system directory:

[Unit]
Description=Btrfs Snapshot Service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/btrfs-snapshot.sh

The btrfs-snapshot.sh script will contain the logic for creating a new snapshot and removing old ones. For example:

#!/bin/bash

# Create a new snapshot
btrfs subvolume snapshot /mnt/btrfs/root /mnt/btrfs/snapshots/$(date +'%Y%m%d')

# Remove old snapshots
find /mnt/btrfs/snapshots -type d -mtime +7 -exec rm -rf {} \;

This script creates a new snapshot with the current date and removes any snapshots older than 7 days. In practice, you may need to adjust the script to fit your specific use case.

Log Rotation

To prevent log files from growing indefinitely, we can use log rotation tools like logrotate. I usually create a file called btrfs-snapshot in the /etc/logrotate.d directory:

/mnt/btrfs/snapshots/*.log {
    daily
    missingok
    notifempty
    delaycompress
    compress
    maxsize 100M
    maxage 7
    postrotate
        /usr/sbin/service btrfs-snapshot restart > /dev/null
    endscript
}

This configuration will rotate the log files in the /mnt/btrfs/snapshots directory daily, compressing them and keeping them for up to 7 days. This is where people usually get burned - forgetting to set up log rotation can lead to disk space issues down the line.

Troubleshooting

If you encounter issues with your Btrfs snapshots, there are several things you can check. First, make sure that the btrfs-snapshot.timer and btrfs-snapshot.service units are enabled and running:

systemctl status btrfs-snapshot.timer
systemctl status btrfs-snapshot.service

You can also check the system logs for any errors related to the snapshot service:

journalctl -u btrfs-snapshot.service

Additionally, you can use the btrfs command to check the status of your snapshots:

btrfs subvolume list /mnt/btrfs

This command will list all the subvolumes on your Btrfs file system, including the snapshots.

Best Practices

To avoid issues with Btrfs snapshots, it’s essential to follow best practices. First, make sure to regularly clean up old snapshots to prevent them from filling up your disk. You can use the find command to remove old snapshots, as shown in the btrfs-snapshot.sh script. Second, make sure to monitor your disk space and adjust your snapshot retention policy accordingly. You can use tools like df and du to monitor your disk space.

For more information on Btrfs and its features, you can visit the Btrfs wiki or the kernel.org website.