Set Up SSH Keys on Ubuntu 24.04 — Secure Access in 5 Minutes

Introduction

SSH key authentication replaces password-based login with a cryptographic key pair — a private key stored on your local machine and a public key stored on your server. When you connect, the server challenges your client to prove it holds the matching private key. No password crosses the network, and brute-force attacks become effectively impossible.

Every Raff tutorial lists "SSH access configured" as a prerequisite, and SSH keys are the foundation of that access. Password authentication is vulnerable to dictionary attacks, credential stuffing, and shoulder surfing. SSH keys eliminate all three. Once configured, logging in is also faster — no typing passwords, no two-factor prompts for every connection.

In this tutorial, you will generate an Ed25519 SSH key pair on your local machine, copy the public key to your Raff Ubuntu 24.04 VM, verify key-based login works, disable password authentication to lock down the server, and troubleshoot common connection issues. The entire process takes under five minutes. We use Ed25519 because it produces shorter, faster keys with security equivalent to RSA 4096-bit — it is the recommended algorithm on all modern systems.

Step 1 — Generate an SSH Key Pair on Your Local Machine

Run this command on your local computer (not the server). If you are on Linux, macOS, or Windows 10+ with OpenSSH, the ssh-keygen command is already available.

Generate an Ed25519 key pair:

bashssh-keygen -t ed25519 -C "your_email@example.com"

The -t ed25519 flag selects the Ed25519 algorithm. The -C flag adds a comment (typically your email) to help identify the key later.

If you need to connect to older systems that do not support Ed25519 (rare in 2026, but possible), generate an RSA key instead:

bashssh-keygen -t rsa -b 4096 -C "your_email@example.com"

For all other cases, Ed25519 is the better choice — shorter keys, faster authentication, and equivalent security.

You will see:

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/your_user/.ssh/id_ed25519):

Press Enter to accept the default location (~/.ssh/id_ed25519). If you already have a key at this path, ssh-keygen will ask whether to overwrite it — type n and choose a different filename if you want to keep the existing key.

Next, you are prompted for a passphrase:

Enter passphrase (empty for no passphrase):

A passphrase encrypts your private key on disk. If someone steals your laptop, they cannot use the key without the passphrase. Enter a strong passphrase and confirm it, or press Enter twice to skip (less secure but more convenient for automated scripts).

The output confirms your key pair was created:

Your identification has been saved in /home/your_user/.ssh/id_ed25519
Your public key has been saved in /home/your_user/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx your_email@example.com

You now have two files:

• ~/.ssh/id_ed25519 — Your private key. Never share this file. Never copy it to a server.
• ~/.ssh/id_ed25519.pub — Your public key. This is what you place on servers you want to access.

Step 2 — Copy the Public Key to Your Server

The easiest way to install your public key on the server is ssh-copy-id. This command connects to your server via password, creates the ~/.ssh/authorized_keys file if it does not exist, and appends your public key.

From your local machine, run:

bashssh-copy-id your_user@your_server_ip

Replace your_user with your username on the server and your_server_ip with your Raff VM's public IP address.

You will be prompted for your server password:

your_user@your_server_ip's password:

Enter your password. On success, you will see:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'your_user@your_server_ip'"
and check to make sure that only the key(s) you wanted were added.

If ssh-copy-id is not available on your system (some minimal Windows installations), you can copy the key manually:

bashcat ~/.ssh/id_ed25519.pub | ssh your_user@your_server_ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

This one-liner creates the .ssh directory with correct permissions, appends your public key to the authorized_keys file, and sets the file permissions to owner-only read/write.

Step 3 — Verify Key-Based Authentication

Test the connection from your local machine:

bashssh your_user@your_server_ip

If your key has a passphrase, you will be prompted to enter it. If the key has no passphrase, you will be logged in immediately with no password prompt at all.

You can also verify which authentication method was used by checking the verbose output:

bashssh -v your_user@your_server_ip 2>&1 | grep "Authentication succeeded"

You should see Authentication succeeded (publickey) confirming key-based login was used instead of a password.

If you still see a password: prompt, key authentication is not working. Check these common causes:

• Permissions on the server — The .ssh directory must be 700 and authorized_keys must be 600:

bashchmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

• Ownership — The .ssh directory and its contents must be owned by your user:

bashchown -R your_user:your_user ~/.ssh

• Correct key offered — If you used a non-default filename, specify it explicitly:

bashssh -i ~/.ssh/your_custom_key your_user@your_server_ip

Once you confirm key-based login works, proceed to the next step.

Step 4 — Disable Password Authentication

With key-based authentication verified, disable password login entirely. This prevents any brute-force or dictionary attack from succeeding, even if an attacker has unlimited time.

On your server, open the SSH daemon configuration:

bashsudo nano /etc/ssh/sshd_config

Find and modify these directives. Some may be commented out — remove the # and set the value:

PasswordAuthentication no
KbdInteractiveAuthentication no
UsePAM yes

Setting PasswordAuthentication no disables password-based login. KbdInteractiveAuthentication no disables keyboard-interactive authentication which can also prompt for passwords on some systems.

Ubuntu 24.04 also has an SSH config drop-in directory. Check for any override files that might re-enable password authentication:

bashls /etc/ssh/sshd_config.d/

If you see a file like 50-cloud-init.conf, open it and ensure it does not contain PasswordAuthentication yes. If it does, change it to no or delete the line.

Restart the SSH service to apply the changes:

bashsudo systemctl restart ssh

Test from a new terminal window on your local machine:

bashssh your_user@your_server_ip

You should be logged in via key. To confirm password auth is truly disabled, try connecting with a forced password prompt:

bashssh -o PubkeyAuthentication=no your_user@your_server_ip

You should see Permission denied (publickey) — this confirms the server no longer accepts passwords.

Step 5 — Configure SSH for Convenience (Optional)

If you manage multiple Raff VMs, you can create an SSH config file on your local machine to simplify connections.

Create or edit the SSH config file on your local machine:

bashnano ~/.ssh/config

Add an entry for your server:

Host myserver
    HostName your_server_ip
    User your_user
    IdentityFile ~/.ssh/id_ed25519

Save and close the file. Set the correct permissions:

bashchmod 600 ~/.ssh/config

Now you can connect with just:

bashssh myserver

Add more Host blocks for each Raff VM you manage. You can also set options like ServerAliveInterval 60 to prevent idle disconnections on long sessions, or Port 2222 if you have changed the default SSH port on a particular server.

Conclusion

You have generated an Ed25519 SSH key pair, installed the public key on your Raff Ubuntu 24.04 VM, verified key-based login, and disabled password authentication. Your server now only accepts cryptographic key-based access — the strongest practical authentication method for SSH.

From here, you can:

• Configure the UFW firewall to further restrict access to your server
• Harden your server with additional security measures like fail2ban and automatic updates
• Secure Nginx with Let's Encrypt for encrypted web traffic alongside your encrypted SSH access
• Upload your SSH public key in the Raff dashboard so new VMs are provisioned with your key automatically — no password login needed from the first boot

Combined with UFW rate limiting on port 22 and Raff's built-in DDoS protection, SSH key authentication forms a strong security foundation for any workload. On our own infrastructure, we enforce key-only access on every server — password authentication is disabled from the first boot. Our security team recommends this as the first step after provisioning any new VM.

If you ever need emergency access to a locked-out server, the Raff web console provides browser-based terminal access directly through the hypervisor, bypassing SSH entirely. This safety net means disabling password authentication carries no risk of permanent lockout.