DevOps is hard

See which process is occupying a port — February 14, 2021
SSH Multiplexing and Master Mode — January 31, 2021

SSH Multiplexing and Master Mode

When using SSH bastion hosts it is common to set up new connections for many of the use cases discussed in the previous section throughout the day.
Normally we would start a new TCP connection for each one of them. However, open TCP connection are a finite resource on any machine, and each one of them takes some time to set up.

Multiplexing is a feature provided by SSH which alleviates these problems. It allows a single TCP connection to carry multiple SSH sessions. The TCP connection will be established and kept alive for a specific period of time and new SSH sessions will be established over that connection.

It works by creating a “control socket” file which will be used every time we want start a new connection.

We need to pass two command line arguments in order to leverage this feature:

Continue reading
SSH X11 Forwarding —

SSH X11 Forwarding

If you are using SSH between Unix-like operating systems, you can also forward GUI applications over SSH. This is especially useful if your server doesn’t really have a user interface, but you need to check something on the fly with a web browser running on it.

Descarga Navegador Firefox — Rápido, privado y gratis — de Mozilla

This is possible because all Unix-like systems share a common GUI windowing system called X11, which is what provides the basic framework for the desktop environment: drawing and moving windows on the display device and interacting with a mouse and keyboard, etc.

Continue reading
SSH Tunnelling and Port Forwarding —

SSH Tunnelling and Port Forwarding

In the previous section we saw how to make use of a jump host as a proxy to run commands into a remote machine.
Sometimes however, having a shell is not necessary, and the connectivity aspect of having a secure channel to the remote host is way more interesting.
SSH’s port forwarding feature allows us to create a secure channel to the remote host, and then use it to carry any type of traffic back and forth.

Local Port Forwarding

Scenario: you want to reach a specific application (which listens on a specific port) on the remote server.

For example (as in the image) we might want to connect to a PostgreSQL database application that is listening on port 5432 on the remote server, which is in a private subnet.

Continue reading
Jumping SSH Hosts —

Jumping SSH Hosts

Sometimes some servers are unreachable to us due to network topology barriers, which are put in place for security reasons.

A common way to allow developers and sysadmins access is to provide public “bastion” or jump hosts, where one can obtain a shell and then use it to connect to one’s destination.

Looking at the picture above, things may seem pretty simple – but how is authentication handled when more than two parties are involved in a connection?

Continue reading
SSH Config —

SSH Config

As well as having to type passphrases, remembering the right user, key file, port and other settings for each host you want to SSH into might quickly become overwhelming.

That’s why, in addition to the command line options we saw so far, SSH also allows to read the same options from configuration files.

The configuration data will be applied from the following sources in the following order:

  1. command-line options
  2. user’s configuration file (located in ~/.ssh/config)
  3. system-wide configuration file (located in /etc/ssh/ssh_config)

How to write ~/.ssh/config and /etc/ssh/ssh_config files

Continue reading
SSH Agent —

SSH Agent

If you are using key pair based authentication with a passphrase for your keys, things can quickly get tedious as you have to input the passphrase every time you want to connect somewhere. If you want to avoid that, you can optionally use another preinstalled tool: ssh-agent.

The ssh-agent is a little helper program that keeps track your identity keys and their passphrases. The agent is consulted by the SSH client during the authentication process instead of the user having to specify a key – and having to type its passphrase all over again.

Adding keys

Simply add your private key file to the agent like this

Continue reading
SSH Known Hosts —

SSH Known Hosts

Much like how the authorized_keys file is used to authenticate clients on the server, there is another file in the ~/.ssh folder called known_hosts, which is used to authenticate servers to the client.

Whenever SSH is configured on a new server it always generates a public and private key pair for the server, just like you did for your user in the previous section. Every time you connect to any SSH server, it shows you its public key first, together with a proof that it possesses the corresponding private key. If you do not have its public key yet, then your computer will ask for it and add it into the known_hosts file. 

This way, the client can check that the server is a known one, and not some rogue server trying to pass off as the right one.

That’s why when you connect to a server for the first time, you might get a message like this:

Continue reading
SSH Authentication methods —

SSH Authentication methods

There are two ways of authenticating to a server with SSH: user/password based authentication (which is now by many considered outdated and insecure) and key pair based authentication. Let’s start from the legacy one and build up to the modern way of doing things:

Username and Password Based Authentication

In this version, all you will need to to connect to your server is to run

Continue reading
[Series] SSH for Developers – Beyond the Basics — January 25, 2021

[Series] SSH for Developers – Beyond the Basics

I have written this guide for developers who might be superficially familiar with the basics of SSH – maybe even fiddled with a file or two without really knowing what’s going on – and would like a more cohesive overview of its most powerful features.

This is essentially an organised collection of the main gotchas and topics that gave me (a dev, not a sysadmin) the most dexterity in jumping about from machine to machine in the cloud.
It is meant to be read in succession as each topic builds on the previous, but I also tried to keep them loosely coupled in case the reader is only interested in a particular section.

Have fun.

Table of Contents:

  1. Authentication
  2. Known Hosts
  3. SSH Agent
  4. Config
  5. Jumping Hosts
  6. Tunnelling and Port Forwarding
  7. X11 Forwarding
  8. Multiplexing and Master Mode

Disclaimer: This does not mean to be a complete list of all that is possible with SSH. That would probably take a book. Or two. But any feedback about additions is welcome.

First, a refresher. What is SSH?

SSH is a protocol built on top of TCP that is intended to provide a secure channel for a client and a server to communicate. In its implementation, it consists of two programs:

Continue reading