Articles

Design Production gRPC Architecture with Load Balancing and TLS (2026)

April 18, 2026 9 min read

A gRPC service, when built for production, isn’t just a set of RPC methods; it’s a distributed system where client and server communication needs to be robust, secure, and scalable. The core challenge is ensuring that requests are distributed efficiently across multiple server instances, and that the communication itself is protected from eavesdropping and tampering.

Let’s imagine a simple gRPC service, say, a Greeter service with a SayHello RPC.

syntax = "proto3";

package greeter;

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

On the server side, you might have several instances of your gRPC server running, perhaps on different machines or different ports on the same machine.

// server.go (simplified)
package main

import (
	"context"
	"log"
	"net"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

	pb "your_module/greeter" // Assuming your generated pb package
)

type server struct {
	pb.UnimplementedGreeterServer
}

func (s *server) SayHello(ctx context.Context, req *pb.HelloRequest) (*pb.HelloReply, error) {
	log.Printf("Received: %v", req.GetName())
	if req.GetName() == "invalid" {
		return nil, status.Errorf(codes.InvalidArgument, "name cannot be 'invalid'")
	}
	return &pb.HelloReply{Message: "Hello " + req.GetName()}, nil
}

func main() {
	lis, err := net.Listen("tcp", ":50051") // Example port
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterGreeterServer(s, &server{})
	log.Printf("server listening at %v", lis.Addr())
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

And on the client side:

// client.go (simplified)
package main

import (
	"context"
	"log"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure" // For now, we'll use insecure

	pb "your_module/greeter"
)

const (
	address = "localhost:50051" // Direct address to one server
)

func main() {
	conn, err := grpc.Dial(address, grpc.WithTransportCredentials(insecure.NewCredentials()))
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewGreeterClient(conn)

	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "World"})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	log.Printf("Greeting: %s", r.GetMessage())
}

This setup works for a single instance. But in production, you’ll have multiple instances of your Greeter server running, perhaps on ports 50051, 50052, 50053, etc. How does the client know which one to talk to, and how do you ensure requests are spread out? This is where load balancing comes in.

Load Balancing with gRPC

gRPC has built-in support for client-side load balancing. Instead of directly dialing a single server address like "localhost:50051", you dial a logical address. This logical address is often a DNS name that resolves to multiple IP addresses, or a custom resolver that provides a list of server addresses.

The client library then uses this list of addresses to distribute requests. The default load balancing policy in gRPC is round_robin. This means it will cycle through the available server addresses for each new RPC.

To enable client-side load balancing, you need to:

  1. Configure the client to use a resolver: The client needs to know how to get the list of backend server addresses. This is done via a resolver.Builder. For DNS-based resolution, you can use grpc.DefaultResolverConfig with a DNS resolver.
  2. Provide a list of server addresses: The resolver will fetch these. If you’re using DNS, the DNS record for your service (e.g., greeter.example.com) should resolve to multiple A or AAAA records, each pointing to a different server instance.
  3. Specify a balancer (optional but common): While round_robin is the default, you can configure other policies like pick_first (always try the first server, then failover) or custom ones.

Here’s how you’d modify the client to use a DNS-based resolver and round-robin balancing. First, ensure your DNS entry for greeter.example.com resolves to multiple IPs for your server instances.

// client_lb.go (simplified)
package main

import (
	"context"
	"log"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"
	"google.golang.org/grpc/resolver" // Import resolver package

	pb "your_module/greeter"
)

const (
	// Use a logical DNS name for the service.
	// The DNS A/AAAA records for greeter.example.com should point to your server IPs.
	target = "dns:///greeter.example.com:50051" // The port is usually the same for all instances
)

func main() {
	// Register a resolver for DNS. gRPC has a built-in DNS resolver.
	// This step is often implicit if you're using the default resolver.
	// If you need to explicitly configure, you might do something like:
	// grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`)

	// Dial the logical target. The client will resolve this and manage connections.
	conn, err := grpc.Dial(target,
		grpc.WithTransportCredentials(insecure.NewCredentials()), // Still insecure for now
		grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`), // Explicitly set policy
	)
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewGreeterClient(conn)

	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "LoadBalancedWorld"})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	log.Printf("Greeting: %s", r.GetMessage())
}

When this client runs, grpc.Dial will use the default DNS resolver to look up greeter.example.com. It will get a list of IP addresses and then use the round_robin policy to connect to and distribute requests among the available servers. If a server becomes unhealthy, the resolver will eventually update the list, and the load balancer will stop sending traffic to it.

Securing Communication with TLS

Plain gRPC traffic is unencrypted. For production, you absolutely need TLS to encrypt data in transit and authenticate both the client and server.

To enable TLS, you need:

  1. Server Certificates: Each gRPC server instance needs a TLS certificate and a corresponding private key. These should be signed by a Certificate Authority (CA) that your clients trust. For internal services, you might run your own CA.
  2. Client Trust: Clients need to trust the CA that signed the server certificates. This is typically done by providing the CA certificate to the client when establishing the TLS connection.
  3. gRPC TLS Configuration: Both client and server need to be configured to use TLS.

On the server side, you’ll use grpc.NewServer with grpc.Creds configured for TLS.

// server_tls.go (simplified)
package main

import (
	"context"
	"crypto/tls"
	"log"
	"net"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/credentials" // Import credentials
	"google.golang.org/grpc/status"

	pb "your_module/greeter"
)

// ... (server struct and SayHello method as before) ...

func main() {
	// Load server certificate and key
	cert, err := tls.LoadX509KeyPair("server.crt", "server.key")
	if err != nil {
		log.Fatalf("failed to load key pair: %v", err)
	}

	// Configure TLS
	tlsConfig := &tls.Config{
		Certificates: []tls.Certificate{cert},
		// MinVersion: tls.VersionTLS12, // Good practice to enforce minimum version
	}
	grpcTLSCreds := credentials.NewTLS(tlsConfig)

	// Create gRPC server with TLS credentials
	s := grpc.NewServer(grpc.Creds(grpcTLSCreds))

	lis, err := net.Listen("tcp", ":50051")
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	pb.RegisterGreeterServer(s, &server{})
	log.Printf("server listening at %v with TLS", lis.Addr())
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

On the client side, you’ll use grpc.Dial with grpc.WithTransportCredentials configured for TLS.

// client_tls.go (simplified)
package main

import (
	"context"
	"log"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials" // Import credentials
	"google.golang.org/grpc/credentials/tls" // Import TLS credentials

	pb "your_module/greeter"
)

const (
	target = "greeter.example.com:50051" // Using DNS name for LB and TLS
)

func main() {
	// Load CA certificate to verify the server's certificate
	caCert, err := credentials.NewClientTLSFromFile("ca.crt", "") // "" means no specific server name override
	if err != nil {
		log.Fatalf("failed to load CA certificate: %v", err)
	}

	// Dial with TLS credentials
	conn, err := grpc.Dial(target,
		grpc.WithTransportCredentials(caCert), // Use the CA cert for verification
		grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
	)
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewGreeterClient(conn)

	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "TLS-SecuredWorld"})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	log.Printf("Greeting: %s", r.GetMessage())
}

Combining Load Balancing and TLS

The real power comes when you combine these. You’d use a DNS name for the target in your grpc.Dial call. This DNS name resolves to multiple server IPs, enabling load balancing. Then, you provide TLS credentials to grpc.Dial to secure the communication. The gRPC client will automatically select a server from the resolved list, establish a TLS connection to it, and then perform the RPC.

The grpc.DefaultServiceConfig is a powerful mechanism. It allows you to embed load balancing policies and other client-side configurations directly into the target string or as a default for all dials. For example, you can specify the loadBalancingPolicy and even configure specific resolvers or balancer configurations.

// client_combined.go (simplified)
package main

import (
	"context"
	"log"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"

	pb "your_module/greeter"
)

const (
	// Target uses DNS for resolution (load balancing) and specifies the service name.
	// TLS will be handled by the credentials.
	target = "greeter.example.com:50051"
)

func main() {
	// Load CA certificate to verify the server's certificate
	caCert, err := credentials.NewClientTLSFromFile("ca.crt", "")
	if err != nil {
		log.Fatalf("failed to load CA certificate: %v", err)
	}

	// Dial with both TLS credentials and a service config for load balancing.
	conn, err := grpc.Dial(target,
		grpc.WithTransportCredentials(caCert),
		grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
	)
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewGreeterClient(conn)

	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "ProdReadyClient"})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	log.Printf("Greeting: %s", r.GetMessage())
}

This combined approach ensures that your gRPC calls are distributed across available server instances and that the data exchanged is encrypted and authenticated.

A key detail often overlooked is how the load balancer interacts with TLS. When a client connects to greeter.example.com:50051, the DNS resolver might return multiple IPs. The round_robin policy will pick one IP. The client then attempts to establish a TLS handshake with the server at that IP. If the handshake succeeds, the RPC proceeds. If the server at that IP is down or misconfigured (e.g., wrong certificate), the client will fail the connection and the load balancer will mark that backend as unhealthy, eventually trying another IP from the resolved list. The resolver and balancer work in concert to maintain a healthy pool of backends for the client.

The next challenge is managing server-side load balancing and health checking, which often involves external components like a load balancer (e.g., Nginx, HAProxy, or cloud provider LBs) in front of your gRPC servers, or using service discovery and health checking mechanisms like Consul or etcd.

Want structured learning?

Take the full Grpc course →