Turn /tmp into a tmpfs so you never run out of space during a log surge -

Why /tmp on a tmpfs is a lifesaver for log‑heavy workloads

If a service starts spamming logs, a full /tmp can bring the whole system to a halt. On low‑RAM servers or in containers where /tmp lives on a regular disk partition, the problem is even worse. Mounting /tmp as a tmpfs keeps it in memory, guarantees instant cleanup, and stops disk‑space exhaustion. Below is a practical, step‑by‑step guide that covers the trade‑offs and security notes you’ll need to keep in mind.

1. Inspect the current layout

df -hT /tmp

Typical output on a stock Debian 12:

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4  100G  45G   50G  48% /
tmpfs          tmpfs  2G     0  2G   0% /tmp

If /tmp is a regular partition, the tmpfs line will be missing. Pay attention to the size reported; it’s usually a fraction of RAM (often 1/4).

2. Decide on the size

tmpfs grows lazily, but you should cap it to avoid exhausting RAM. A common rule of thumb is RAM / 4. For a 16 GiB server:

TMPFS_SIZE=$((16 * 1024 / 4))M   # 4096M

If you have swap, you can push the limit higher, but be careful: swapping a large tmpfs can degrade performance.

3. Create a systemd unit for the mount

Modern distributions use systemd for mounts. Create /etc/systemd/system/tmp.mount:

[Unit]
Description=Temporary Directory
Documentation=man:tmpfs(5)
Before=local-fs.target

[Mount]
What=tmpfs
Where=/tmp
Type=tmpfs
Options=mode=1777,size=4096M

[Install]
WantedBy=local-fs.target

mode=1777 gives the usual sticky‑bit permissions.
size= caps the memory usage.
Before=local-fs.target ensures it mounts before services that might write to /tmp.

Reload systemd and enable the unit:

sudo systemctl daemon-reload
sudo systemctl enable --now tmp.mount

Verify:

mount | grep /tmp

You should see:

tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=4194304k,mode=1777)

4. Persist across reboots

The unit file is already persistent. If you prefer to use /etc/fstab, add:

tmpfs   /tmp   tmpfs   defaults,size=4096M,mode=1777   0   0

But the systemd approach is cleaner and integrates with systemctl status tmp.mount.

5. Adjust services that rely on /tmp

Some daemons create large temporary files (e.g., systemd-journald, apache, nginx, docker). Make sure they’re configured to use /tmp or a subdirectory that will be cleaned automatically.

Apache: TempDir /tmp in httpd.conf.
Docker: --tmpfs /tmp:rw,size=1g in daemon.json.
systemd-journald: Storage=volatile keeps logs in RAM; otherwise, logs are written to /var/log/journal.

If a service writes to /tmp but you need persistence across reboots, consider a dedicated tmpfs mount for that service (e.g., /var/tmp).

6. Security considerations

Isolation: tmpfs is shared across all users. If a process writes a malicious file to /tmp, any other user can read it. Mitigate by using --private-tmp in systemd service files or by mounting a separate tmpfs for sensitive services.
```
[Service]
PrivateTmp=true
```
Sticky bit: The default mode=1777 protects against accidental deletion. Keep it.
No persistence: Data in /tmp disappears on reboot. If you need to keep logs, configure services to write to /var/log instead.
Swap leakage: If the tmpfs is swapped out, sensitive data could end up on disk. Disable swap for critical systems or use swapoff -a before mounting a large tmpfs.

7. Troubleshooting

Symptom	Likely cause	Fix
`systemd` refuses to start a service that writes to `/tmp`	Service is not allowed to use `/tmp` due to `PrivateTmp=true`	Disable `PrivateTmp` or adjust the service’s configuration to use a different directory
`/tmp` fills up with large files	Service writes more than expected	Increase `size=` or move the service’s temp directory to a dedicated tmpfs
Memory pressure spikes after heavy log activity	Tmpfs is too large	Reduce `size=` or add swap
Unexpected file permissions in `/tmp`	Mount options overridden by `systemd` unit	Verify `Options=` in the unit file

8. Real‑world example: protecting a log‑surge from a misbehaving web app

A Node.js app on a 4 GiB VM started generating 500 MiB of logs per minute. The /tmp partition was 512 MiB, so the VM crashed after 10 minutes. After switching /tmp to a 1 GiB tmpfs:

sudo systemctl edit tmp.mount

Add:

[Mount]
Options=mode=1777,size=1024M

Reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart tmp.mount

The VM stayed alive, and the logs were still accessible in /var/log/nodeapp. The app’s temporary files were now in RAM, preventing disk exhaustion.

9. When not to use tmpfs for /tmp

Large, long‑lived data: If a service legitimately needs to keep large files across reboots, a disk partition is required.
Very low RAM: On a 512 MiB device, a 128 MiB tmpfs may still be too large.
Security‑critical environments: If you cannot tolerate the risk of shared memory, use PrivateTmp or separate tmpfs mounts per service.

10. Automation with Ansible

For repeatable deployments, an Ansible role can enforce the tmpfs mount:

- name: Ensure /tmp is a tmpfs
  ansible.builtin.mount:
    path: /tmp
    src: tmpfs
    fstype: tmpfs
    opts: "mode=1777,size=4096M"
    state: mounted
    fstab: yes

Add this to your host group and run ansible-playbook -i inventory site.yml.

11. Keep an eye on memory usage

free -h shows the tmpfs usage under Mem: Used. If you see a sudden spike, investigate which process is writing to /tmp:

sudo lsof +D /tmp | awk '{print $1, $2, $9}' | sort | uniq -c | sort -nr

This command lists processes by the number of open files in /tmp.

12. Summary of trade‑offs

Benefit	Drawback
Immediate cleanup	Data lost on reboot
No disk I/O	Consumes RAM
Simple configuration	Requires careful size tuning
Works with systemd	Potential swap leakage

When you need a fast, fail‑safe /tmp that never runs out of space during a log surge, a tmpfs mount is the most straightforward solution. It blends seamlessly with systemd, keeps the system responsive, and aligns with a security‑aware mindset by allowing per‑service isolation when needed.