MLOps Observability: Logs, Metrics, Traces for ML (2026)

MLOps observability isn’t about seeing your ML system; it’s about predicting its future behavior based on its past.

Let’s look at a model in production, serving predictions for a recommendation engine. We’ll use a simplified Python example with Flask for the API and a placeholder for our model.

from flask import Flask, request, jsonify
import time
import random

app = Flask(__name__)

# Simulate model loading
model_version = "v2.1"
print(f"Loading model: {model_version}")
# In a real scenario, this would load a trained model artifact

@app.route('/predict', methods=['POST'])
def predict():
    start_time = time.time()
    data = request.get_json()
    user_id = data.get('user_id')
    num_recommendations = data.get('num_recommendations', 5)

    # Simulate model inference
    # In a real scenario, this would be model.predict(features_for_user_id)
    predictions = [f"item_{random.randint(1000, 9999)}" for _ in range(num_recommendations)]
    prediction_latency = random.uniform(0.05, 0.2) # Simulate ML model inference time
    time.sleep(prediction_latency)

    end_time = time.time()
    total_latency = end_time - start_time

    # Simulate logging
    print(f"Request received: user_id={user_id}, num_recommendations={num_recommendations}")
    print(f"Inference latency: {prediction_latency:.4f}s, Total latency: {total_latency:.4f}s, Model: {model_version}")
    print(f"Predictions: {predictions}")

    return jsonify({
        'user_id': user_id,
        'predictions': predictions,
        'model_version': model_version,
        'latency_ms': int(total_latency * 1000)
    })

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

When you send a POST request like this:

curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"user_id": "user_123", "num_recommendations": 3}'

The system executes a series of steps: receiving the request, extracting data, performing inference (simulated), and returning a response. Observability tools help us instrument and visualize this entire flow.

Logs are the raw, unstructured (or semi-structured) text records of events. They tell you what happened.

• Purpose: Debugging individual requests, understanding specific errors, auditing.
• Example: The print statements in the Flask app are basic logs. In production, you’d use structured logging (e.g., JSON) for easier parsing.
• Collection: Tools like Fluentd, Logstash, or cloud-native services (CloudWatch Logs, Stackdriver Logging) collect and centralize logs.

Metrics are aggregated, numerical measurements over time. They tell you how well things are performing.

• Purpose: Monitoring system health, performance trends, alerting on deviations.
• Example: Request rate, error rate, average inference latency, CPU/memory usage of the model server.
• Collection: Libraries like Prometheus client, StatsD, or built-in cloud provider metrics expose these. Prometheus scrapes these endpoints, and Grafana visualizes them.

Traces provide a end-to-end view of a single request’s journey across multiple services or components. They tell you where time was spent.

• Purpose: Understanding latency bottlenecks, visualizing distributed system interactions, identifying dependencies.
• Example: A trace for /predict would show the time spent in the web server, the ML model inference, and any downstream calls the model might make.
• Collection: Standards like OpenTelemetry or OpenTracing, with agents/libraries (e.g., Jaeger, Zipkin clients) embedded in your application code.

The real power comes from combining these. You might see a spike in latency (metric), then drill down to the traces for requests during that spike to see which specific model inference calls are slow, and then examine the logs for those slow requests to find an underlying error or resource constraint.

The most surprising thing is how often the simplest ML models exhibit the most complex observability challenges, not because their internal logic is intricate, but because their dependencies and data drift are harder to track.

When you’re instrumenting your ML model for inference, beyond just logging the input and output, you must also capture the version of the model that generated the prediction. This is critical for debugging data drift issues; if a prediction seems wrong, you need to know which model produced it to correlate it with training data or feature distributions from that specific time.

The next frontier is correlating these signals with model performance on unseen data and business outcomes.