Linux Security

Rolling Back a Broken Flatpak Update on a Production Server

Flatpak is great for sandboxed desktop apps, but on a production box you’re still dealing with real‑world workloads. A bad runtime bump can bring an entire workflow to a halt or expose a vulnerability if the new runtime regresses. The silver lining? Flatpak keeps a full history of every branch you’ve ever installed, so you can undo a bad update without touching other apps.

Ever had a host you added to /etc/hosts just stop answering? I’ve seen this happen after a kernel upgrade or a quick edit that left a stray comment. On Fedora, Debian, and Arch the culprit is almost always the local resolver: systemd‑resolved. It only reads /etc/hosts if the ResolveFile option is enabled and the service is started with --resolve. A tiny mis‑config can silence that hook.

1. Check that systemd‑resolved is actually running

systemctl status systemd-resolved.service

If you see inactive or failed, start it up:

When you have to touch a host that lives behind several bastions, forwarding the agent is the quickest way to keep your private key off every hop. The catch? A compromised bastion can steal that key if you’re not careful. Below is a hands‑on walk‑through that shows how to tighten forwarding, what to watch out for, and how to double‑check that the key never ends up where it shouldn’t.

1. Understand the Threat Model

The aim is to confine the key to the exact hosts you trust and to stop accidental leaks.

When a user or a process tries to run something with sudo, the action is recorded in the system journal. Because sudo writes to the journal, you can filter on the SYSLOG_IDENTIFIER or MESSAGE fields to spot failed attempts, audit who tried what, and even trigger alerts.

Below is a step‑by‑step guide that shows how to:

• Identify failed sudo attempts in real time
• Pull historical data for a specific user or time window
• Distinguish between legitimate and suspicious activity
• Automate the process with a small script
• Understand trade‑offs and common pitfalls

The examples assume a recent systemd‑based distro (Debian 12, Ubuntu 24.04, Fedora 40, etc.) and that sudo is compiled with the default --syslog flag (enabled since 2019). If your system uses a custom sudo build or a different logging backend, the commands will need adjustment.

SSH jump host silently dropping connections after OpenSSH 9.5

A recent OpenSSH 9.5 release tweaked how the server handles TCP keepalive packets. My jump host – a modest Ubuntu 24.04 box – started dropping established SSH sessions after a few minutes of idle time. The problem popped up during a routine audit when several users complained of “connection closed” messages, yet nothing appeared in the logs.

What changed in 9.5?

OpenSSH 9.5 introduced new defaults for ClientAliveInterval and ClientAliveCountMax on the server side:

systemd‑resolved is the default DNS resolver on most modern distros. It pulls /etc/hosts unless you flip the --no-hosts flag. In 2026 a subtle tweak in its default behaviour started swallowing local hostname lookups, and no one noticed until a service went down.

Why it happens

systemd-resolved now gives the DNS stack a higher priority than the hosts file when you call getaddrinfo(3). If a name shows up in both /etc/hosts and a DNS zone, the DNS answer wins. When the zone is empty or the query times out, it silently falls back to the hosts file. But if the DNS server is unreachable and the hosts entry points to 127.0.0.1 or ::1, the resolver drops that entry and returns ENOTFOUND. The change was meant to stop a compromised DNS server from poisoning local lookups.

When a host suddenly resolves a domain to an old IP, the first instinct is to restart NetworkManager or reboot the machine. That works, but it’s heavy‑handed and can disrupt services. In a production environment, a brief outage can trigger SLA violations or expose stale data to clients. Below is a concise, hands‑on guide to flushing the systemd‑resolved cache safely, with a focus on security‑aware operators who need minimal downtime.

Why ss Might Show No Open TCP Ports Even When Services Are Running

I’ve seen this happen on fresh installs, on servers that have just rebooted, and even on production machines that have been up for months. The symptom is simple: ss -tuln prints nothing, but systemctl status nginx (or whatever daemon you’re checking) says active (running). The mismatch usually boils down to one of three things:

1. The service is listening on a non‑TCP socket
2. The service is bound to a Unix domain socket or a local‑only address
3. The socket is managed by a socket‑activated unit that isn’t yet in use

Below is a quick checklist that will help you zero in on the culprit, plus a few hardening tips that come with each case.

HTTP is the backbone of most web services. When a client can’t reach a web server, the first instinct is to look at the application logs, DNS, or the network path. A silent drop by nftables is a common, yet often overlooked, culprit. This post walks through a systematic approach to uncovering why nftables is silently discarding HTTP packets, with a focus on the port‑filtering rules that most administrators set up for hardening.

I upgraded the kernel on my 2026‑era Debian‑based home‑lab server from 6.1.12‑generic to 6.5.0‑generic. The installer was a breeze – “All packages up to date” and a clean reboot. A few hours later, Nginx stopped answering HTTP. The systemctl status nginx looked like this:

● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Mon 2026‑11‑06 14:32:12 UTC; 1h 12min ago
  Process: 1234 ExecStart=/usr/sbin/nginx -g daemon off; (code=exited, status=0/SUCCESS)
   Main PID: 1234 (code=exited, status=0/SUCCESS)

The master exited cleanly, yet the port was closed. The only clue? The kernel had changed. Below is the debugging walk‑through, the root cause, and how to harden the system so a kernel upgrade never silently kills a critical service again.