On a busy host, a 1‑second DNS lookup can feel like a blink, but when it happens on every request, it adds up. In 2026, most distributions ship systemd‑resolved as the default resolver, and the default configuration is usually fine. Yet a handful of real‑world scenarios—high‑traffic web servers, containers, or networks with strict firewall rules—expose a subtle 1‑second delay that can be traced back to systemd‑resolved’s handling of upstream DNS servers.
This post walks through the symptoms, the root cause, and practical fixes that keep DNS fast without compromising security.
Why a 1‑second delay matters
- HTTP latency – A single 1‑second DNS round‑trip can push a page load from 200 ms to 1.2 s, hurting user experience.
- API rate limits – Some services throttle requests that exceed a certain latency threshold; a slow DNS can trigger those limits.
- Container orchestration – Kubernetes, Docker Compose, and other orchestrators perform many DNS lookups per pod start; a 1‑second stall can delay cluster boot.
- Security hardening – A slow resolver can mask DNS amplification attacks or make it harder to detect malicious queries.
1. Identify the symptom
The classic symptom is a consistent 1‑second delay when resolving a domain that is known to exist on the local network or in a trusted DNS zone.
$ time dig @127.0.0.53 example.local
Typical output:
; <<>> DiG 9.18.5 <<>> @127.0.0.53 example.local
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12345
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;example.local. IN A
;; ANSWER SECTION:
example.local. 60 IN A 192.168.1.42
;; Query time: 1021 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon Nov 25 12:34:56 UTC 2026
;; MSG SIZE rcvd: 56
The Query time: 1021 msec indicates the resolver spent a full second waiting for a response. The same delay appears for any domain that is not cached locally.
2. Root cause: “fallback‑timeout” and “cache‑size” mis‑tuning
systemd‑resolved has two key settings that influence this behavior:
| Setting | Default | Effect |
|---|---|---|
CacheSize | 1024 | Number of cached entries. |
FallbackTimeout | 1 s | Time to wait for a reply from the primary server before falling back to the secondary. |
When the primary DNS server is unreachable (e.g., firewall blocks UDP 53, or the server is down), systemd‑resolved waits FallbackTimeout before querying the secondary. If the secondary is also unreachable, the resolver will retry the primary after the same timeout, causing a 1‑second stall on every lookup.
Common triggers
- Firewall rules – Many hardened setups block outbound UDP/TCP 53 to reduce DNS amplification risk. If the firewall blocks the primary but not the secondary, the resolver will stall.
- IPv6-only upstream – If the primary is IPv6 and the host has no IPv6 connectivity, the resolver will wait for a timeout before falling back to IPv4.
- Container networking – Docker’s default network isolates DNS traffic; the host’s resolver may try to reach the host’s DNS server over a bridged interface that is blocked.
3. Quick sanity check
$ systemd-resolve --status | grep -i 'DNS Servers'
If you see two servers and one is unreachable, that’s the likely culprit.
$ ping -c 1 -4 8.8.8.8
If the ping fails, the firewall is blocking outbound UDP/TCP 53.
4. Fixes
4.1. Adjust FallbackTimeout
If you know the primary server is unreliable, reduce the timeout to 0.1 s. Add or edit the following in /etc/systemd/resolved.conf:
[Resolve]
FallbackTimeout=100ms
Reload:
sudo systemctl restart systemd-resolved
Trade‑off: A shorter timeout may cause more frequent fallbacks, which can increase DNS traffic. Use only if you’re certain the primary is often unreachable.
4.2. Reorder DNS servers
Place the most reliable server first. In /etc/systemd/resolved.conf:
[Resolve]
DNS=10.0.0.1 10.0.0.2
The resolver will try 10.0.0.1 first; if it fails, it will immediately fall back to 10.0.0.2.
4.3. Disable the fallback entirely
If you have a single, highly reliable DNS server, you can disable the fallback:
[Resolve]
DNS=10.0.0.1
FallbackDNS=
This removes the 1‑second delay because the resolver never waits for a second server.
4.4. Use systemd-resolve --flush-caches
Sometimes the cache holds stale negative responses. Flush it:
sudo systemd-resolve --flush-caches
4.5. Verify firewall rules
Open outbound UDP/TCP 53 to the primary server:
sudo iptables -A OUTPUT -p udp --dport 53 -d 10.0.0.1 -j ACCEPT
sudo iptables -A OUTPUT -p tcp --dport 53 -d 10.0.0.1 -j ACCEPT
If you use nftables:
sudo nft add rule inet filter output ip daddr 10.0.0.1 udp dport 53 accept
sudo nft add rule inet filter output ip daddr 10.0.0.1 tcp dport 53 accept
Security note: Opening DNS traffic is a common attack vector. Restrict the destination to the known DNS server(s) and consider DNSSEC validation (DNSSEC=yes in resolved.conf).
4.6. Container‑specific workaround
For Docker, set the DNS server explicitly in the daemon configuration:
{
"dns": ["10.0.0.1"]
}
Then restart Docker:
sudo systemctl restart docker
This bypasses the host’s resolver and eliminates the fallback delay inside containers.
5. Advanced tuning: CacheSize and CacheNegativeTTL
If you have a high‑traffic environment, you might want to increase the cache size to reduce lookups:
[Resolve]
CacheSize=4096
CacheNegativeTTL=30
A larger cache keeps more positive and negative responses, cutting down on repeated queries. The negative TTL controls how long a failed lookup stays cached; a shorter TTL means the resolver will retry sooner, which can help if the upstream becomes available quickly.
6. Monitoring and alerting
Add a simple check to your monitoring stack:
#!/usr/bin/env bash
dig @127.0.0.53 example.local +time=1 +short | grep -q 192.168.1.42
if [[ $? -ne 0 ]]; then
echo "DNS stall detected" | mail -s "systemd-resolved stall" admin@example.com
fi
Schedule it with cron or a systemd timer. This gives you an early warning if the resolver starts stalling again.
7. Security implications
- DNSSEC – Enabling DNSSEC (
DNSSEC=yes) forces the resolver to validate signatures, which adds a small CPU cost but protects against spoofed responses. It does not affect the 1‑second stall unless the upstream server is unreachable. - Split DNS – In environments with internal and external zones, ensure the internal DNS server is reachable. A misconfigured split DNS can trigger the fallback delay.
- Firewall hardening – Restrict outbound DNS to known servers. A permissive rule (
-p udp --dport 53 -j ACCEPT) opens a channel for amplification attacks. Use the destination‑specific rules shown above.
8. Checklist for a production system
| Item | Command / Config | Why |
|---|---|---|
| Verify DNS servers | systemd-resolve --status | Ensure primary is reachable |
Adjust FallbackTimeout | /etc/systemd/resolved.conf | Reduce stall |
| Reorder servers | /etc/systemd/resolved.conf | Reliable first |
| Flush cache | systemd-resolve --flush-caches | Remove stale negatives |
| Harden firewall | iptables/nftables | Prevent amplification |
| Monitor stalls | cron script | Early detection |
| Enable DNSSEC | DNSSEC=yes | Protect against spoofing |
9. Real‑world example
A mid‑size web farm ran into a 1‑second DNS stall after a firewall upgrade that blocked outbound UDP 53 to the corporate DNS. The load balancer’s health checks started failing, and the application layer reported increased latency.
Fix applied:
- Opened UDP/TCP 53 to the corporate DNS in the firewall.
- Set
FallbackTimeout=100msto avoid unnecessary waits. - Reordered DNS servers to put the corporate DNS first.
- Enabled DNSSEC for added integrity.
Result: DNS queries resolved in under 10 ms, and the application latency dropped from 1.2 s to 200 ms.
10. Takeaway
A 1‑second delay in systemd‑resolved is usually a symptom of a mis‑configured fallback path or a firewall rule that blocks the primary DNS server. By inspecting systemd-resolve --status, adjusting FallbackTimeout, ordering DNS servers, and tightening firewall rules, you can eliminate the stall without compromising security. Remember to monitor the resolver and keep DNSSEC enabled for integrity.
See also
- Application-Level Firewalls
- Resolve DNS Cache Poisoning on Raspberry Pi with resolvectl – One‑Step Fix
- Fixing a Crashed Btrfs Subvolume After Unexpected Reboot
- When /etc/hosts Won’t Resolve: A Quick Systemd‑Resolved Fix
- Recovering a Kernel Panic After Adding a New Initramfs Module
- Multi‑jump bastion access without storing keys: practical ssh‑agent forwarding