Linux Server Hardening Checklist

A default Linux install is not a secure Linux install. Misconfigurations like open root SSH access, weak firewalls, and unpatched software account for the majority of breaches. New servers face automated scans within minutes of going online, so hardening should happen before anything else.

This checklist covers the core steps: locking down SSH, configuring firewalls, patching, tightening file permissions, enabling mandatory access controls, and setting up audit logging.

Lock Down SSH

SSH is your primary access point and the first thing attackers probe. The default config (password auth, root login, port 22) is exactly what automated scanners look for.

Generate an Ed25519 key pair, which offers better security and performance than RSA:

ssh-keygen -t ed25519

Once key-based login works, update /etc/ssh/sshd_config:

PasswordAuthentication no
ChallengeResponseAuthentication no
PubkeyAuthentication yes
PermitRootLogin no
AllowUsers yourname
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2

Change the default port from 22 to something less obvious. This won't stop a determined attacker, but it cuts noise from automated scans significantly.

Always test changes from a second terminal before closing your current session. Run sudo sshd -t to check for syntax errors, then systemctl reload sshd to apply without dropping active connections.

Add Two-Factor Authentication

2FA means an attacker needs both your SSH key and physical access to your device. Install the Google Authenticator PAM module:

sudo apt install libpam-google-authenticator   # Debian/Ubuntu
sudo dnf install google-authenticator           # RHEL/Fedora

Run google-authenticator for each user to generate a secret key and recovery codes. Store the recovery codes offline.

Add this line to /etc/pam.d/sshd:

auth required pam_google_authenticator.so

Then update /etc/ssh/sshd_config:

KbdInteractiveAuthentication yes
UsePAM yes
AuthenticationMethods publickey,keyboard-interactive

Keep an active session open while testing. TOTP codes depend on accurate system time, so make sure NTP is running.

Configure Firewalls and Fail2Ban

Run a host-based firewall even if your server sits behind a network firewall. The principle is simple: deny all incoming traffic by default, then allow only what you need.

For Ubuntu/Debian (UFW):

ufw default deny incoming
ufw default allow outgoing
ufw limit ssh
ufw enable

For RHEL/Rocky/AlmaLinux (Firewalld):

firewall-cmd --set-default-zone=public
firewall-cmd --permanent --add-service=ssh
firewall-cmd --permanent --add-service=https
firewall-cmd --reload

Harden the kernel network stack by adding these to /etc/sysctl.conf:

net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1

Install Fail2Ban

Fail2Ban monitors login attempts and bans IPs after repeated failures. Create /etc/fail2ban/jail.local (don't edit jail.conf directly, updates will overwrite it) and configure it to ban IPs for one hour after three failed attempts within 10 minutes. Set the correct backend for your firewall (banaction = ufw or banaction = nftables).

Audit Services and Remove Legacy Protocols

Check what's listening with ss -tlnp and what's running with systemctl list-units --type=service --state=running. Disable anything you don't need: Bluetooth, CUPS, avahi-daemon, rpcbind.

Remove legacy protocols that transmit data in cleartext:

On Debian/Ubuntu: sudo apt-get --purge remove xinetd nis tftpd telnetd rsh-server. On RHEL-based systems: yum erase xinetd ypserv tftp-server telnet-server rsh-server. Verify removal with ss -tulpn.

Patch and Automate Updates

Updating your system is the fastest way to close known security gaps. Run updates immediately after provisioning:

apt update && apt upgrade -y      # Debian/Ubuntu
dnf update -y                      # RHEL/Rocky

Then automate security patches. On Debian/Ubuntu, install unattended-upgrades and configure it to apply security patches only. On RHEL/Rocky, install dnf-automatic and set upgrade_type = security in /etc/dnf/automatic.conf.

Set up email notifications for update results. Disable automatic reboots on production servers (Automatic-Reboot = false) so reboots happen during planned maintenance windows. For high-uptime environments, consider live patching with Canonical Livepatch (Ubuntu) or kpatch (RHEL).

Harden File Systems and Permissions

Audit SUID and SGID binaries first. These files run with elevated privileges and are prime targets for exploitation:

find / -xdev \( -perm -4000 -o -perm -2000 \) -type f -ls

Tighten permissions on critical files: /etc/shadow should be 600, /etc/passwd should be 644, /etc/ssh/sshd_config should be 600. Set a global umask of 027 in /etc/profile to prevent new files from being world-readable.

Find and fix world-writable files with find / -xdev -type f -perm -0002 -ls. For directories that must stay world-writable (like /tmp), apply the sticky bit: chmod 1777 /tmp.

Secure Mount Options

Edit /etc/fstab to restrict what can happen on critical partitions:

Test changes with mount -o remount before rebooting to avoid boot issues.

Enable Mandatory Access Controls

SELinux and AppArmor add kernel-level restrictions on what processes can do. Use whichever your distribution ships with: SELinux for RHEL/CentOS/Fedora, AppArmor for Ubuntu/Debian/SUSE. Switching between them causes compatibility issues.

SELinux: Check status with getenforce. Start in permissive mode (setenforce 0) for at least two weeks to capture workload behaviour without breaking anything. Monitor violations with ausearch -m avc -ts recent. Use audit2why to diagnose blocks and audit2allow -M [module_name] to create policy modules. Once logs are clean, switch to enforcing with setenforce 1, then make it permanent in /etc/selinux/config.

AppArmor: Check active profiles with aa-status. Install apparmor-utils for management commands. Start profiles in complain mode with aa-complain, then move to enforce mode with aa-enforce once you're confident. Use aa-genprof to build profiles for custom applications.

Set Up Audit Logging and Monitoring

Without logging, incidents leave no trace. Install auditd:

sudo apt-get install auditd audispd-plugins

Add file system watches for critical files:

-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity

Track all root-level command execution:

-a always,exit -F arch=b64 -S execve -F euid=0 -k root_commands

Load rules with augenrules --load and add -e 2 at the end of your rules file to make the config tamper-proof (changes require a reboot).

File Integrity Monitoring with AIDE

AIDE detects unauthorized file changes by comparing the current state against a known-good baseline. Install it, initialise the database with aideinit, and move the resulting file to /var/lib/aide/aide.db.gz. Set up a daily cron job to run aide --check and email results to administrators.

Centralise Logs

Local logs are useless if an attacker with root access deletes them. Forward logs to a remote server in real time using rsyslog with TLS encryption. Add to /etc/rsyslog.conf:

*.* @@remote-host:514

Set LogLevel VERBOSE in your SSH config so logs include key fingerprints for every successful login. For production environments managing multiple servers, tools like Wazuh or OSSEC provide host-based intrusion detection with centralised log analysis.

Ongoing Maintenance

Hardening is not a one-time task. Configurations drift, new vulnerabilities appear, and staff changes leave orphaned accounts behind.

Weekly: Review Fail2Ban logs, check for failed updates, verify backups.

Monthly: Audit user accounts and permissions, review running services, run a full scan with Lynis or OpenSCAP.

Quarterly: Rotate credentials, update firewall rules, test disaster recovery.

Use infrastructure-as-code tools like Ansible with dev-sec.io hardening roles to enforce consistent configurations across your fleet and prevent drift between audits.

FDC's dedicated servers give you full root access and complete control over your security stack. Explore dedicated server options to build on a platform where you control every layer.