The most surprising thing about setting up your own OpenVPN server is how little the server itself actually does during a connection; it’s mostly just a glorified router with a fancy authentication layer.
Let’s see it in action. Imagine you’re on your laptop, outside your home network, trying to connect to your OpenVPN server running on a Raspberry Pi.
First, your laptop (the client) generates a public/private key pair. It sends the public key to the OpenVPN server. The server, which already has its own public/private key pair, verifies the client’s public key against a pre-shared list of trusted client certificates. If it matches, the server sends back its own certificate (containing its public key). Your client then uses the server’s public key to encrypt a symmetric session key. This encrypted session key is sent to the server. The server uses its private key to decrypt the session key, and now both client and server share a secret symmetric key. This key is used for all subsequent communication, providing confidentiality and integrity.
The problem this solves is giving you a secure, encrypted tunnel back into your home network from anywhere in the world. Without it, accessing your home files or even just browsing the internet from a public Wi-Fi hotspot is risky. OpenVPN allows you to create this private network extension, making it seem like your laptop is physically present on your home network.
Internally, OpenVPN uses a client-server model. The server listens on a specific UDP or TCP port (commonly UDP 1194). Clients initiate connections to this port. The core of the setup involves generating certificates and keys for both the server and each client using a Certificate Authority (CA). The server needs its own certificate and key, and each client needs a certificate and key signed by the same CA. This establishes trust.
The configuration files are critical. On the server side, server.conf (or similar) dictates its role. Key directives include port 1194, proto udp, dev tun (for a routed IP tunnel), ca ca.crt, cert server.crt, key server.key, dh dh.pem, and server 10.8.0.0 255.255.255.0 to define the VPN subnet. push "route 192.168.1.0 255.255.255.0" would push your home network’s route to clients, allowing them to access devices on your LAN.
On the client side, client.conf mirrors much of this but specifies the server’s public IP address or hostname (remote vpn.yourdomain.com 1194), and crucially, includes the client’s certificate and key: ca ca.crt, cert client.crt, key client.key. The tls-auth or tls-crypt directive adds an extra layer of security, using a pre-shared key to authenticate the TLS handshake itself, making it harder for attackers to even attempt a handshake.
The exact levers you control are primarily within these configuration files. The cipher and auth directives determine the encryption and authentication algorithms used for the tunnel traffic (e.g., cipher AES-256-GCM, auth SHA256). The keepalive directive ensures the tunnel stays open by sending periodic pings. The client-to-client directive, if enabled on the server, allows connected VPN clients to see each other directly, which is useful for some scenarios but a security risk if not needed.
When you configure dev tun, OpenVPN creates a virtual network interface on both the client and server, acting like a point-to-point link. Traffic destined for the VPN subnet (e.g., 10.8.0.0/24) is routed through this tun interface. If you use dev tap, it creates an Ethernet tunnel, broadcasting layer 2 traffic, which is more complex and often unnecessary. The server directive on the server configures it to hand out IP addresses from the specified VPN subnet to connecting clients.
A crucial, often overlooked part of the server setup is enabling IP forwarding on the server’s operating system. Without this, the server won’t know how to route traffic originating from the VPN clients to your actual home network, or vice-versa. On Linux, this is typically done by uncommenting net.ipv4.ip_forward=1 in /etc/sysctl.conf and running sudo sysctl -p.
The next concept you’ll likely grapple with is managing multiple clients and their certificates, specifically how to revoke access for a lost or compromised device.