What Is SSH? Explained Simply (2026)

Every DevOps engineer uses SSH daily. Most have a vague understanding of how it works. Here's the complete picture — simple and practical.

What SSH Is

SSH (Secure Shell) is a protocol that lets you connect to a remote computer over a network — and control it as if you were sitting in front of it.

Before SSH, engineers used Telnet. Telnet sent everything — including passwords — in plain text over the network. Anyone with a packet sniffer could read your credentials.

SSH encrypts everything. The connection is secure even on untrusted networks.

SSH is how you:

• Connect to an AWS EC2 instance
• Deploy code to a server
• Run commands on a remote Kubernetes node
• Transfer files between machines (using SCP or SFTP, which run over SSH)

How SSH Works (Simply)

SSH uses asymmetric cryptography — a pair of mathematically linked keys:

• Private key — stays on your laptop, never shared with anyone
• Public key — placed on the server you want to connect to

When you connect, the server uses your public key to create a challenge that only your private key can solve. If your key solves it — you're authenticated. No password sent over the network.

This is why SSH keys are more secure than passwords.

Your First SSH Connection

Generate an SSH key pair:

# Generate a 4096-bit RSA key pair
ssh-keygen -t ed25519 -C "your-email@example.com"
 
# Or RSA (older, also fine)
ssh-keygen -t rsa -b 4096 -C "your-email@example.com"
 
# This creates:
# ~/.ssh/id_ed25519      ← private key (NEVER share this)
# ~/.ssh/id_ed25519.pub  ← public key (this goes on servers)

Connect to a server:

# Basic connection
ssh username@server-ip-or-hostname
 
# With specific private key
ssh -i ~/.ssh/my-key.pem ubuntu@ec2-12-34-56-78.compute-1.amazonaws.com
 
# With specific port (default is 22)
ssh -p 2222 username@server-ip

For AWS EC2:

# Amazon Linux
ssh -i my-key.pem ec2-user@<public-ip>
 
# Ubuntu
ssh -i my-key.pem ubuntu@<public-ip>
 
# Debian
ssh -i my-key.pem admin@<public-ip>

The ~/.ssh/ Directory

ls -la ~/.ssh/
 
# id_ed25519        → your private key
# id_ed25519.pub    → your public key
# authorized_keys   → public keys of people allowed to SSH into this machine
# known_hosts       → fingerprints of servers you've connected to
# config            → SSH client configuration (shortcuts)

authorized_keys is the critical file on the server side. To allow someone to SSH into your server, add their public key to this file:

# On the server
echo "ssh-ed25519 AAAA...their-public-key... their-email" >> ~/.ssh/authorized_keys
 
# Set correct permissions (SSH is strict about this)
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

The SSH Config File (~/.ssh/config)

Instead of typing long SSH commands, define shortcuts:

# ~/.ssh/config

Host prod-server
    HostName 12.34.56.78
    User ubuntu
    IdentityFile ~/.ssh/prod-key.pem
    Port 22

Host dev-server
    HostName 98.76.54.32
    User ec2-user
    IdentityFile ~/.ssh/dev-key.pem

Host bastion
    HostName bastion.mycompany.com
    User deploy
    IdentityFile ~/.ssh/company-key.pem
    ForwardAgent yes

Now instead of:

ssh -i ~/.ssh/prod-key.pem ubuntu@12.34.56.78

You just type:

ssh prod-server

SSH Key Permissions (Common Error)

SSH is strict about key file permissions. If permissions are too open, SSH refuses to use the key:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'my-key.pem' are too open.

Fix:

chmod 600 ~/.ssh/my-key.pem     # Private key: read/write for owner only
chmod 644 ~/.ssh/my-key.pem.pub # Public key: readable by all (fine)
chmod 700 ~/.ssh/               # SSH directory: owner only

Useful SSH Commands

Copy files with SCP:

# Copy file from local to remote
scp my-file.txt ubuntu@server-ip:/home/ubuntu/
 
# Copy file from remote to local
scp ubuntu@server-ip:/var/log/app.log ./app.log
 
# Copy entire directory
scp -r ./my-folder ubuntu@server-ip:/home/ubuntu/
 
# With SSH config shortcut
scp my-file.txt prod-server:/home/ubuntu/

Port forwarding (tunneling):

# Access a remote service on your local machine
# E.g., access remote PostgreSQL (port 5432) at localhost:5433
ssh -L 5433:localhost:5432 ubuntu@server-ip
 
# Access Kubernetes dashboard through a bastion
ssh -L 8001:localhost:8001 bastion-host

Run a single command without interactive shell:

ssh ubuntu@server-ip "df -h"
ssh ubuntu@server-ip "docker ps -a"
ssh ubuntu@server-ip "tail -f /var/log/app.log"

SSH Agent — avoid typing passphrase repeatedly:

# Start agent
eval $(ssh-agent)
 
# Add your key (once per session)
ssh-add ~/.ssh/id_ed25519
 
# Now SSH won't ask for passphrase
ssh prod-server

Jump Hosts / Bastion Servers

In production, servers often don't have public IPs — you connect through a bastion host (jump server):

Your laptop → Bastion (public IP) → Private server (no public IP)

Single jump:

ssh -J bastion-user@bastion-ip target-user@private-server-ip

In ~/.ssh/config:

Host private-server
    HostName 10.0.1.50
    User ubuntu
    ProxyJump bastion
    IdentityFile ~/.ssh/private-key.pem

Host bastion
    HostName bastion.mycompany.com
    User ec2-user
    IdentityFile ~/.ssh/bastion-key.pem

Then just: ssh private-server

SSH Security Best Practices

Disable password authentication (most important):

On the server, edit /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin no

Then restart SSH: sudo systemctl restart sshd

Change the default port:

Port 2222  # in /etc/ssh/sshd_config

Reduces automated bot attacks on port 22 (not security by obscurity — just noise reduction).

Use fail2ban:

# Install on Ubuntu
sudo apt install fail2ban
 
# Automatically bans IPs with too many failed login attempts

Rotate keys regularly: Generate new keys, add to servers, remove old keys from authorized_keys.

SSH in the Cloud

AWS EC2:

• AWS creates a key pair for you (or you upload your own)
• The public key goes into the instance at launch
• You connect with: ssh -i your-key.pem ec2-user@<ip>
• AWS Systems Manager Session Manager is an alternative that doesn't require SSH at all

GitHub/GitLab: Add your public key to your account settings. All git operations then use SSH instead of HTTPS password authentication:

# Test GitHub SSH connection
ssh -T git@github.com
# Hi username! You've successfully authenticated...

SSH in One Sentence

SSH is an encrypted tunnel between your terminal and a remote machine — secured by a key pair where your private key never leaves your laptop.

If you understand key pairs, authorized_keys, and the config file — you know 90% of what you need for daily DevOps work.