A service mesh doesn’t magically make your distributed system resilient; it simply moves the complexity of inter-service communication from your application code into a dedicated infrastructure layer.
Let’s see what that looks like. Imagine two microservices, frontend and backend, running in Kubernetes. Without a service mesh, frontend needs to know backend’s IP address and port, handle retries if backend is slow, and potentially encrypt traffic itself.
# frontend deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
replicas: 2
selector:
matchLabels:
app: frontend
template:
metadata:
labels:
app: frontend
spec:
containers:
- name: frontend
image: my-frontend-app:latest
ports:
- containerPort: 8080
---
# backend deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 3
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: backend
image: my-backend-app:latest
ports:
- containerPort: 5000
Now, let’s introduce Istio, a popular service mesh. We’ll install it into the istio-system namespace.
# Install Istio (example command, actual command depends on version and installation method)
curl -L https://istio.io/downloadIstio | sh -
cd istio-1.22.1 # or your downloaded version
bin/istioctl install --set profile=default -y
We’ll then enable Istio injection for the default namespace where our services reside. This is typically done by labeling the namespace.
kubectl label namespace default istio-injection=enabled
After this, Kubernetes will automatically inject a sidecar proxy (Envoy) into every pod in the default namespace. The frontend and backend pods will now have two containers: your application container and the istio-proxy container.
kubectl get pods -n default
# Example output:
# NAME READY STATUS RESTARTS AGE
# frontend-abcde-fghij 2/2 Running 0 1m
# backend-klmno-pqrst 2/2 Running 0 1m
Notice the 2/2 READY status – that’s your app container and the Istio proxy.
With the sidecar in place, all network traffic entering or leaving your frontend pod is intercepted by its istio-proxy. Similarly, traffic to and from backend is intercepted. The sidecars talk to each other, managed by Istio’s control plane (istiod).
Now, frontend doesn’t need to know backend’s IP. It can simply make a request to backend.default.svc.cluster.local (the Kubernetes DNS name for the service). The istio-proxy in the frontend pod intercepts this request, consults istiod for routing rules, and forwards the request to one of the healthy backend pods.
Let’s define a Kubernetes Service for backend if you don’t have one already:
apiVersion: v1
kind: Service
metadata:
name: backend
labels:
app: backend
spec:
ports:
- port: 5000
targetPort: 5000
protocol: TCP
name: http
selector:
app: backend
And a corresponding Istio VirtualService to define how traffic should be routed to backend:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: backend
spec:
hosts:
- backend
http:
- route:
- destination:
host: backend
port:
number: 5000
This VirtualService tells Istio that any traffic directed to the backend host (which resolves to the Kubernetes Service) should be routed to the Kubernetes Service named backend on port 5000. The istio-proxy handles the actual IP resolution and load balancing.
This is where the magic appears to happen. Your frontend application code remains blissfully unaware of the network complexity. It just makes an HTTP request. The sidecar proxy handles:
- Service Discovery: Finding available
backendinstances. - Load Balancing: Distributing requests across
backendpods. - Retries & Timeouts: Automatically retrying failed requests or timing out slow ones.
- Traffic Shifting: Gradually rolling out new versions of
backendby sending a percentage of traffic to them. - Mutual TLS (mTLS): Encrypting all traffic between services without application code changes.
- Observability: Generating metrics, logs, and traces for every request.
The control plane, istiod, is the brain. It configures all the sidecar proxies based on the Istio resources you define (like VirtualService, DestinationRule, Gateway).
# Example DestinationRule for configuring retries and circuit breakers
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: backend
spec:
host: backend
trafficPolicy:
connectionPool:
tcp:
maxConnections: 100
http:
http1MaxPendingRequests: 100
http2MaxRequests: 100
outlierDetection:
consecutive5xxErrors: 3
interval: 10s
baseEjectionTime: 30s
maxEjectionPercent: 50
tls:
mode: ISTIO_MUTUAL # Enables mTLS
This DestinationRule tells the sidecar proxies how to handle connections to backend. It configures connection limits and outlierDetection (a form of circuit breaking) to automatically remove unhealthy backend pods from the load balancing pool if they start returning too many 5xx errors. The tls: mode: ISTIO_MUTUAL setting enforces mTLS for all traffic to backend.
The most surprising thing about a service mesh is that it doesn’t actually solve distributed system problems; it merely abstracts them away from your application code, making them an infrastructure concern. The underlying challenges of network unreliability, latency, and failure modes remain, but they are now managed by a separate, configurable layer.
Consider this: if your backend service is experiencing high latency, your frontend application still experiences high latency. However, with a service mesh, you can now diagnose this by looking at the metrics generated by the istio-proxy sidecar in the frontend pod, without needing to instrument your frontend application code itself. You can also configure advanced resilience patterns like circuit breakers and sophisticated retry logic directly in Istio configuration, affecting all services that communicate with backend, not just frontend.
One thing many people don’t realize is that the sidecar proxy adds a small amount of latency to every single request. While often measured in milliseconds, it’s a constant overhead. For ultra-low-latency applications, this added hop can be significant. The proxy also consumes CPU and memory resources within each pod, which needs to be accounted for in your cluster’s resource planning.
The next logical step after mastering basic routing and resilience is managing ingress and egress traffic with Istio Gateways.