Harden a Fresh VPS in 15 Minutes: SSH Keys, Firewall, Fail2ban and Auto-Updates

Here's something that surprises people the first time. You spin up a brand-new VPS, and before you've installed a single thing, strangers are already knocking on its door.

Nobody picked your server. Bots just sweep through huge ranges of IP addresses, looking for open SSH ports and weak passwords. A fresh server is an easy target because it's still running the defaults.

The good news is that a short checklist closes most of those doors. It takes about 15 minutes, and all you need is a terminal.

This guide is written for Ubuntu 22.04 or newer and Debian 12 or newer. The order of the steps matters, so I'll point out where it does.

What you'll need: a fresh VPS, its IP address, a way to log in (root or a default sudo user), and the location of your provider's web console. If you lock yourself out, that console is how you get back in.

In the commands below, replace YOUR_SERVER_IP with your server's address and deploy with the username you want.

The plan

Here's the whole checklist at a glance.

Each step adds a layer, and an attacker has to get past all of them.

flowchart LR
    A["Bots and scanners"] --> B["UFW<br/>only needed ports open"]
    B --> C["sshd<br/>keys only, no root login"]
    C --> D["Fail2ban<br/>bans repeat offenders"]
    D --> E["Auto-updates<br/>known holes get patched"]
    E --> F["Your server"]

No single layer is perfect. That's the point of having several.

Step 1: Update the system

Log in to your new server first:

ssh root@YOUR_SERVER_IP

Some providers give you a default user like ubuntu instead of root. If that's you, put sudo in front of each command below, for example sudo apt update && sudo apt full-upgrade -y.

Now update everything:

apt update && apt full-upgrade -y

On Ubuntu, a screen may pop up asking which services to restart. That's normal. Accept the defaults and carry on.

Then check whether the updates need a reboot (this usually means a new kernel):

[ -f /var/run/reboot-required ] && echo "Reboot needed"

If it says so, run reboot, wait a minute, and log back in. It's far easier to reboot now than later, when the server is busy.

Step 2: Create a non-root user

Running everything as root is risky. One typo can do real damage, and root is the first username every bot tries.

Create a normal user and give it sudo rights:

adduser deploy
usermod -aG sudo deploy

adduser asks you for a password. Pick a strong one, because you'll need it every time you use sudo.

Minimal Debian image? sudo might not be installed. Run apt install sudo first, then the usermod command.

Step 3: Set up SSH key login

A password can be guessed. A modern SSH key can't, not in any practical sense. Keys are the single biggest upgrade you can make.

On your own computer (not the server), create a key pair if you don't have one:

ssh-keygen -t ed25519 -C "you@laptop"

When it asks for a passphrase, set one. It protects your key if your laptop is ever stolen.

Now copy the public key to your new user:

ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@YOUR_SERVER_IP

That command needs password login to still be on. If your provider already turned it off (many cloud images do), reuse the key you logged in with. On the server, in your first session:

sudo mkdir -p /home/deploy/.ssh
sudo cp ~/.ssh/authorized_keys /home/deploy/.ssh/authorized_keys
sudo chown -R deploy:deploy /home/deploy/.ssh
sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys

On Windows, ssh-copy-id usually isn't available. In that case, switch to the new user in your first session with sudo su - deploy, and add the key by hand:

mkdir -p ~/.ssh && chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Paste the single line from your .pub file into authorized_keys, then save.

Test it before you go any further

Open a new terminal window and log in as the new user:

ssh deploy@YOUR_SERVER_IP
sudo whoami

You should get in with your key. If you set a passphrase, it asks for that, not the server password. Then sudo whoami asks for the password you chose in Step 2 and prints root. If that all works, you're ready for the next step.

Keep your first session open. It's your safety net.

Step 4: Lock down SSH

Now we tell the SSH server to stop accepting passwords and root logins. This is where most people get tripped up, so read this part carefully.

The trap: the first value wins

OpenSSH reads /etc/ssh/sshd_config, which pulls in every .conf file from /etc/ssh/sshd_config.d/ near the top. And for most settings, sshd keeps the first value it finds, not the last.

That's the opposite of how most config files work.

Cloud images can ship a file called 50-cloud-init.conf that sets PasswordAuthentication yes. If you add your own file called 99-hardening.conf, it gets read later, so it loses. Nothing warns you. Password logins just stay on.

The fix is to name your file so it sorts first. Start by looking at what's already there:

sudo grep -riE 'passwordauthentication|permitrootlogin' /etc/ssh/sshd_config /etc/ssh/sshd_config.d/

Then create your own drop-in file:

sudo nano /etc/ssh/sshd_config.d/00-hardening.conf

Put this inside:

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes

Here's what each line does:

• PermitRootLogin no blocks direct logins as root.

• PasswordAuthentication no turns off password logins, so only keys work.

• KbdInteractiveAuthentication no closes a second route that can also ask for a password.

• PubkeyAuthentication yes makes sure key login stays on.

Before you reload anything, check the file for mistakes:

sudo sshd -t

If it prints nothing, the syntax is fine. Now ask SSH what it will actually use. This is the real proof:

sudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication'

You should see these four values (the order may differ):

permitrootlogin no
passwordauthentication no
kbdinteractiveauthentication no
pubkeyauthentication yes

If any of them is different, another file is winning (or your sshd_config has no Include line, which is rare on recent versions). Go back to the grep command and find it.

Now reload SSH:

sudo systemctl reload ssh

On RHEL-based systems the service is called sshd, so use sudo systemctl reload sshd there.

Here's the safe order for this whole step.

flowchart TD
    A["Create user and add your SSH key"] --> B["Open a second terminal"]
    B --> C{"Key login works for the new user?"}
    C -- Yes --> D["Add the hardening drop-in and reload sshd"]
    C -- No --> E["Fix keys or permissions<br/>(keep the root session open)"]
    E --> B
    D --> F["Test again: key login works,<br/>root and password logins fail"]

Test it from a second terminal

Leave your first session open and test from a new window:

ssh deploy@YOUR_SERVER_IP

That should still work. Now check that the doors are really closed:

ssh root@YOUR_SERVER_IP
ssh -o PubkeyAuthentication=no deploy@YOUR_SERVER_IP

Both should fail with Permission denied (publickey). If they do, you're done with SSH.

Want to go further? You can also allow SSH only from your own IP address. It's a separate topic, but it pairs really well with this checklist.

Step 5: Turn on the firewall

A firewall makes sure that only the ports you choose are reachable. UFW is the friendliest one on Ubuntu and Debian.

Ubuntu includes it already. On Debian, install it first:

sudo apt install ufw

Now set the rules. The order matters here.

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp

If you're running a website on this server, open the web ports too:

sudo ufw allow 80,443/tcp

Only then turn it on:

sudo ufw enable
sudo ufw status verbose

UFW will warn that the command may disrupt existing SSH connections. Because you allowed port 22 first, it's safe to answer y.

Heads up: If you run ufw enable before allowing SSH, you lock yourself out. Allow first, enable second.

UFW also covers IPv6 by default (IPV6=yes in /etc/default/ufw), so you don't need a separate setup for it.

A warning if you use Docker

Docker and UFW don't get along the way you'd expect. When you publish a container port, Docker's traffic goes through a different part of the firewall, and UFW's INPUT rules never see it. A port you thought was blocked can still be open to the world.

The simplest fix is to bind published ports to localhost, and let something like Nginx sit in front:

ports:
  - "127.0.0.1:8080:80"

Other options are filtering in the DOCKER-USER chain, or a helper like ufw-docker.

Step 6: Install Fail2ban

Even with key-only login, bots will keep trying. Fail2ban watches your logs and bans any IP that keeps failing. It cuts the noise and gives you a bit more protection.

sudo apt install fail2ban python3-systemd

The python3-systemd package is what lets Fail2ban read the systemd journal. Installing it explicitly avoids a jail that refuses to start on newer Debian.

Put your settings in jail.local, never in jail.conf, because package updates can overwrite jail.conf:

sudo nano /etc/fail2ban/jail.local

Add this:

[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5
ignoreip = 127.0.0.1/8 ::1 YOUR_HOME_IP

bantime.increment = true
bantime.maxtime   = 1w

[sshd]
enabled = true
backend = systemd

A few notes on this config:

• maxretry = 5 within findtime = 10m means five failures in ten minutes earns a ban.

• ignoreip is your safety valve. Put your own IP there so you can't ban yourself. Replace YOUR_HOME_IP with your real address, or delete it from that line.

• bantime.increment = true makes repeat offenders wait longer each time, up to a week.

• backend = systemd tells Fail2ban to read the system journal.

That last one matters. On recent Debian and Ubuntu server images, rsyslog may not be installed, which means /var/log/auth.log doesn't exist. A jail that waits for that file would silently see nothing. Reading the journal avoids the problem. With this backend, don't add a logpath line. It also needs the python3-systemd package, which is why it's in the install command above.

First, let Fail2ban check your config for mistakes. Then restart it and look at the jail:

sudo fail2ban-client -t
sudo systemctl enable fail2ban
sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

The first command should report that the configuration test was successful. If it complains, read the error before you restart.

You should see the sshd jail listed, with counters for failed attempts and banned IPs.

To make sure your values actually took effect, ask Fail2ban directly:

sudo fail2ban-client get sshd maxretry
sudo fail2ban-client get sshd bantime

If you ever ban yourself by mistake, you can undo it from another connection (or the web console):

sudo fail2ban-client set sshd unbanip YOUR_HOME_IP

Step 7: Turn on automatic security updates

New vulnerabilities show up all the time. A server you only patch when you remember is a server that's out of date.

The unattended-upgrades package installs security updates for you. On Ubuntu it's already installed. On Debian, it's not.

sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

Answer Yes when it asks. Then check the result:

cat /etc/apt/apt.conf.d/20auto-upgrades

You should see something like this:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

To see what it would do, without changing anything, run a dry run:

sudo unattended-upgrade --dry-run --debug

By default it only installs security updates, not every update. That's what you want on a server, because it keeps surprises to a minimum.

One more thing to know: kernel updates only take effect after a reboot, and automatic reboots are off by default. You can either reboot manually every few weeks, or turn on automatic reboots. Rather than editing the big default file, create a small one next to it:

sudo nano /etc/apt/apt.conf.d/52unattended-upgrades-local

Put these two lines inside:

Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";

APT reads these files in alphabetical order, and later ones win. So this file overrides the defaults in 50unattended-upgrades, and it won't be overwritten when the package updates.

Be careful with that on a production box. A reboot at 4 a.m. is fine for a hobby project, but not for a service that can't have downtime.

The final check

Run through this list before you call it done. It takes a minute.

sudo ss -tulpn
sudo ufw status verbose
sudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication'
sudo fail2ban-client status sshd

Here's what you want to see:

• ss shows only SSH (plus ports 80 and 443 if you run a website) listening on all addresses, meaning 0.0.0.0 or [::]. Anything bound to 127.0.0.1 or 127.0.0.53 is local-only and fine. A few system services, like DHCP, may also show up, so look up anything you don't recognize.

• UFW is active, with only the ports you allowed.

• sshd -T says permitrootlogin no and passwordauthentication no.

• The Fail2ban sshd jail is running.

• You can log in as your new user with your key, and root and password logins are refused.

Common mistakes

These are the ones I'd watch for.

• Closing your first session too early. Keep it open until a second terminal can log in.

• Naming the SSH file 99-hardening.conf. The first value wins, so a later file can lose without any warning.

• Enabling UFW before allowing SSH. Allow first, enable second.

• Assuming UFW protects Docker ports. It usually doesn't, unless you set it up to.

• Never rebooting after a kernel update. The patch is installed but not running.

• Not knowing where the web console is. Find it now, while you're calm.

What I left out on purpose

You could do a lot more. These are the things I skip in a first 15 minutes, and why.

Where to go from here

Once the basics are in, a few next steps make sense:

• Allow SSH only from your own IP address.

• Turn on backups or snapshots with your provider, and test a restore.

• Add your provider's cloud firewall as a second layer in front of UFW.

• Set up basic monitoring, so you notice when something looks off.

Wrapping up

None of these steps is hard, and none of them takes long. Together, they take your server from "default and exposed" to "reasonably locked down".

The most important habit is the order. Add a way in, test it from a second window, and only then close the old doors. Keep your first session open, and know where your provider's console is.

Do that, and your fresh VPS stops being an easy target.