Cilium eBPF Networking: Kernel-Level Network Security
The most surprising thing about eBPF is that it allows you to run sandboxed programs directly in the Linux kernel without altering kernel source code or loading kernel modules.
Let’s see what this looks like in practice. Imagine we have a Kubernetes cluster and we want to enforce network policies that are more granular than what Kubernetes NetworkPolicies offer. We want to allow pods to communicate only with specific services on specific ports, and we want to do this at the kernel level for maximum performance and security.
Here’s a basic Cilium setup in a Kubernetes cluster. We’ll use kubectl to apply some configurations.
First, install Cilium itself. This typically involves applying a YAML manifest that deploys the Cilium agent as a DaemonSet on each node and a Kubernetes CRD (Custom Resource Definition) for managing policies.
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.14.0/examples/kubernetes/cilium-clusterrole.yaml
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.14.0/examples/kubernetes/cilium-configmap.yaml
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.14.0/examples/kubernetes/cilium-ds.yaml
Once Cilium is running, you can start defining network policies using the CiliumNetworkPolicy CRD. Unlike Kubernetes NetworkPolicy, CiliumNetworkPolicy can leverage eBPF for efficient enforcement.
Let’s say we have a frontend deployment and a backend deployment. We want the frontend pods to only be able to reach the backend pods on port 8080.
apiVersion: cilium.io/v1alpha1
kind: CiliumNetworkPolicy
metadata:
name: frontend-to-backend
spec:
endpointSelector:
matchLabels:
app: frontend
ingress:
- fromEndpoints:
- matchLabels:
app: backend
toPorts:
- ports:
- port: "8080"
protocol: TCP
When this policy is applied, Cilium’s eBPF programs attached to the network interfaces of the frontend pods will inspect outgoing traffic. If a frontend pod tries to send traffic to a pod that isn’t labeled app: backend or attempts to use a port other than 8080 TCP, the eBPF program will drop the packet before it even leaves the node. This is enforced directly in the kernel’s network stack, bypassing the need for userspace packet filtering.
The mental model for Cilium eBPF networking revolves around visibility and control at the packet level. Cilium uses eBPF programs to hook into various points in the Linux kernel’s networking stack. These programs can inspect, filter, and even modify network packets.
Key components:
- eBPF Programs: Small, sandboxed programs that run in the kernel. Cilium generates these programs dynamically based on your desired policies.
- eBPF Maps: Key-value stores that eBPF programs can use for state and configuration. Cilium uses these maps to store information about endpoints, policies, and connection states.
- Cilium Agent: The userspace daemon running on each node. It watches Kubernetes API for changes to pods and policies, translates them into eBPF programs and maps, and loads them into the kernel.
- Cilium API/CRDs: The interface for defining network policies and other configurations.
When a packet arrives at a node or is about to be sent, an eBPF program attached to a relevant kernel hook (like XDP for ingress or TC for egress) is executed. This program can perform actions based on the packet’s contents and the state stored in eBPF maps. For example, it can check the destination IP and port against a policy, look up the identity of the sender/receiver, and decide whether to allow, drop, or modify the packet.
The exact levers you control are primarily through the CiliumNetworkPolicy (or CiliumClusterwideNetworkPolicy) resources. You define endpointSelector to specify which pods the policy applies to, and then ingress and egress rules to define what traffic is allowed in and out. These rules can be based on labels, IP addresses, CIDRs, ports, and protocols. You also have control over Cilium’s overall behavior through its ConfigMap, such as enabling features like Hubble (for network observability) or defining the CNI backend.
One of the most powerful aspects of eBPF is its ability to provide L7 visibility and control without requiring sidecar proxies. Cilium can parse application-layer protocols like HTTP directly within the kernel using eBPF. This means you can write policies like "allow HTTP GET requests to /api/v1/users from the frontend to the backend" directly in your CiliumNetworkPolicy. The eBPF program intercepts the HTTP request, inspects the method and path, and enforces the policy at that level, all without a traditional proxy sitting in the data path. This dramatically reduces latency and overhead compared to sidecar proxy solutions.
The next concept you’ll likely encounter is distributed tracing and observability with Hubble, which leverages eBPF to provide real-time visibility into your network traffic flows and policy decisions.