gRPC services don’t automatically propagate tracing context across network boundaries, leaving distributed traces fragmented and incomplete.

Let’s see OpenTelemetry in action with gRPC. Imagine a client calling a UserService to get user details, which then calls an OrderService to fetch recent orders. Without tracing, these are separate, unconnected events. With OpenTelemetry, the client initiates a trace, and this trace context (trace ID, span ID of the client’s operation) is injected into the gRPC request headers. The UserService on the server side extracts this context, starts a new span for its own work, and makes it a child of the client’s span. Crucially, when UserService calls OrderService, it injects the current trace context (its own trace ID and its newly created span ID) into the outgoing gRPC request headers. OrderService then extracts this, starts its own child span, and the trace is unified.

Here’s a simplified Go example:

Client-side setup:

import (
	"context"
	"google.golang.org/grpc"
	"google.golang.org/grpc/metadata"
	"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
)

// ... inside your client function
conn, err := grpc.Dial(
	"localhost:50051",
	grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()), // Add client interceptor
	grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()), // Add stream interceptor
)
// ... handle err

client := pb.NewUserServiceClient(conn)
ctx := context.Background()

// The otelgrpc interceptors automatically handle injection/extraction
// when using the standard gRPC client.
resp, err := client.GetUser(ctx, &pb.UserRequest{UserId: "123"})
// ... handle err

Server-side setup:

import (
	"context"
	"google.golang.org/grpc"
	"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
)

// ... inside your server setup
server := grpc.NewServer(
	grpc.UnaryInterceptor(otelgrpc.UnaryServerInterceptor()), // Add server interceptor
	grpc.StreamInterceptor(otelgrpc.StreamServerInterceptor()), // Add stream interceptor
)
// ... register your service

// ... inside your service implementation (e.g., GetUser)
func (s *userServiceServer) GetUser(ctx context.Context, req *pb.UserRequest) (*pb.UserResponse, error) {
	// otelgrpc automatically extracts incoming trace context
	// and creates a new span as a child of the incoming one.
	// You can access the current span if needed:
	// span := trace.SpanFromContext(ctx)
	// span.SetAttributes(attribute.String("user.id", req.UserId))

	// ... your business logic ...

	// If this service calls another gRPC service:
	// You'd create a new gRPC client and the otelgrpc client interceptor
	// would automatically inject the current trace context.
	// ... call orderService.GetOrders(...) ...

	return &pb.UserResponse{Name: "Alice"}, nil
}

The core problem this solves is distributed tracing context propagation. When a request hops between services, especially over network protocols like gRPC, the tracing metadata (like trace ID and parent span ID) needs to be explicitly passed along. OpenTelemetry’s otelgrpc integration does this by hooking into gRPC’s interceptor mechanism. On the client side, it intercepts outgoing requests and injects the current trace context into the outgoing gRPC metadata. On the server side, it intercepts incoming requests, extracts any trace context from the metadata, and uses it to create a new span that is a child of the incoming trace. This ensures that all spans belonging to a single request flow are correctly linked together in your tracing backend.

The otelgrpc interceptors are the key. They abstract away the manual injection and extraction of headers. The UnaryClientInterceptor, StreamClientInterceptor, UnaryServerInterceptor, and StreamServerInterceptor functions provided by the go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc package are what you plug into your grpc.Dial options and grpc.NewServer options, respectively. This automatically handles the underlying W3C Trace Context or B3 propagation formats by embedding the trace information in the grpc.Metadata headers.

What most people miss is that the server-side interceptor itself creates the span for the incoming RPC. You don’t typically need to manually create a span for the top-level RPC handler on the server; otelgrpc.UnaryServerInterceptor() does that for you. Your code within the RPC handler then runs within that automatically created span, and you can add attributes to it or create further child spans for internal operations.

The next concept you’ll likely explore is how to configure the exporter to send these traces to a specific backend like Jaeger or Prometheus, and how to control sampling rates.

Want structured learning?

Take the full Grpc course →