Linux OOM killer tuning for VPS

The Linux OOM killer is the kernel's last resort when memory runs out: it picks a process and terminates it to keep the system alive. On a VPS, where RAM is tight and there's nowhere to fall back to, the default choice is often the wrong one. Your database gets killed, a long-running worker survives, and you're left to figure out why. This guide covers how the OOM killer scores processes, how to bias that scoring towards your critical services, and how to use cgroups so a single runaway process can't take the rest of the system down with it.

How the OOM killer chooses a victim

When the kernel can't reclaim enough memory through page cache eviction or swap, it invokes the OOM killer. Every process has an oom_score between 0 and 1000, derived mostly from its Resident Set Size (RSS) and swap usage. The process with the highest score gets a SIGKILL.

RSS dominates the calculation, which is why the kill almost always lands on the largest memory consumer. That's frequently your database, your application server, or whichever long-lived process is doing the most useful work. The process that actually triggered the allocation, the "invoker", is rarely the one terminated.

There are two flavours of OOM event you need to keep separate:

• Global OOM: the host (or your VPS as a whole) is out of RAM and swap. The kernel scans every process and kills the highest scorer.
• Cgroup OOM: a specific cgroup has hit its memory limit. The kernel only kills inside that cgroup, even if the rest of the system has memory to spare.

If you've configured systemd unit limits or you're running containers, most of your OOM events will be cgroup OOMs. That's a good thing: the blast radius is contained.

Spotting memory pressure before the kill

OOM events are almost never sudden. There's usually a window of growing pressure first, and the goal of monitoring is to catch it inside that window.

free -h gives you the system view. The column that matters is available, not free: it accounts for reclaimable page cache and reflects what you can actually allocate without swapping. Keep MemAvailable above roughly 10 to 15 percent of MemTotal at peak load.

For per-process attribution, sort by RSS:

ps aux --sort=-%mem | head -10

Or use htop and sort by RES. The values you see here feed directly into the kernel's scoring, so the top entries are the most likely OOM targets.

On kernels 4.20 and newer, Pressure Stall Information is the early warning system worth wiring into monitoring:

cat /proc/pressure/memory

The some avg10 figure is the percentage of time at least one task stalled waiting for memory over the last ten seconds. Under 5 percent is fine. Sustained values above 10 percent mean the system is spending real time blocked on memory reclaim, and an OOM kill is plausible.

Swap thrashing shows up in vmstat 1 as non-zero si and so columns sustained over time. A small amount of resident swap is harmless. Constant swap-in and swap-out is not.

Protecting critical processes with oom_score_adj

The score the kernel computes can be biased per-process through oom_score_adj, on a scale from -1000 (immune) to +1000 (kill me first). The adjustment is added directly to the final score.

For a one-shot change against a running process:

echo -500 | sudo tee /proc/$(pidof sshd)/oom_score_adj

For anything you want to persist across restarts, set it in the systemd unit. That's the right place for sshd, your database, and anything else you can't afford to lose:

[Service]
OOMScoreAdjust=-900

Sensible defaults to start from:

• sshd: -1000. If you lose remote access during a memory crisis, recovery gets a lot harder.
• MySQL, PostgreSQL, Redis: -800 to -900. Strong protection without making them completely untouchable in a genuinely catastrophic situation.
• Application workers, batch jobs, cron tasks: +100 to +500. These are the processes you'd rather see killed than your database.

Don't set everything to -1000. If nothing is killable, the kernel will eventually panic or freeze instead, which is worse.

Capping memory with cgroups and systemd

Adjusting scores influences who gets killed. Cgroups influence whether the global kill ever happens. By giving each service a hard upper bound, you push memory failures into a single cgroup rather than letting one process drain the whole VPS.

In a systemd unit file:

[Service]
MemoryHigh=400M
MemoryMax=512M
OOMPolicy=stop
Restart=on-failure
RestartSec=5s

MemoryHigh is a soft throttle: the kernel aggressively reclaims pages from the cgroup above this point but doesn't kill anything. MemoryMax is the hard ceiling. If the cgroup tries to allocate past it, the kernel kills a process inside the cgroup. With Restart=on-failure the service comes straight back up.

On cgroup v2 (Ubuntu 22.04 and later, recent Debian, RHEL 9), memory.oom.group kills every process in the cgroup together instead of leaving orphans behind. Useful for multi-process services like PHP-FPM pools where a half-killed group will misbehave.

A few application-specific notes worth applying:

• PHP-FPM: set pm = ondemand on small VPS instances and size pm.max_children against average per-worker RSS, not the default. A pool sized for 4 GB of headroom on a 2 GB VPS will OOM the first time it fills.
• Node.js: cap the V8 heap with --max-old-space-size=512 (in MB). Without it, Node will happily grow until the kernel intervenes.
• MySQL and PostgreSQL: innodb_buffer_pool_size and shared_buffers should leave clear headroom for the OS page cache, connection memory, and any other tenants on the box. The defaults assume a dedicated server.

Reading the logs after an OOM event

When the OOM killer fires, the kernel dumps a detailed report into the ring buffer. Pull it with:

dmesg -T | grep -iE 'killed process|out of memory'
journalctl -k --grep='Out of memory'

The block to read carefully starts with the invoker and ends with the victim. The kernel prints a full task list with each process's RSS, swap usage, and final oom_score_adj. Three things are worth checking:

• The constraint. CONSTRAINT_NONE means a global OOM, CONSTRAINT_MEMCG means a cgroup hit its limit. The fix is different in each case.
• Free swap. If this is 0kB, both RAM and swap were exhausted. Either add swap, raise MemoryMax on the offender, or reduce concurrency.
• The victim's score versus everything else. If the victim's score isn't much higher than the next few processes, your oom_score_adj values aren't doing enough work. Widen the gap.

For cgroup OOMs specifically, the kill counter lives at memory.events inside the cgroup:

cat /sys/fs/cgroup/system.slice/mysql.service/memory.events

A rising oom_kill count means that service is repeatedly hitting its limit. That's a signal to raise MemoryMax, profile the workload, or move the service to a larger plan, not to keep restarting it on a loop.

Wrapping up

Tuning the OOM killer isn't about making it go away. It's about controlling which process pays the price when memory runs out, and shrinking the blast radius when it happens. The pattern that holds up in production:

• Score-protect the services you can't afford to lose, especially sshd and your databases.
• Cap everything else with MemoryMax in a systemd unit so a single runaway is a single restart, not an outage.
• Watch PSI and MemAvailable rather than waiting for dmesg to tell you the story afterwards.
• Leave 15 to 20 percent of RAM as headroom. Tuning can't compensate for a VPS that's simply too small for the workload.

If your memory pressure is structural rather than configurable, you need more RAM or faster swap-backed storage. FDC Servers' VPS plans run on AMD EPYC with NVMe storage, which keeps swap-backed reads fast enough that short memory bursts don't escalate into kills.