Minikube, Prometheus, and Grafana, when deployed together, form a powerful local monitoring stack, but the most surprising truth is that your local development environment can become more complex and resource-intensive than many production deployments.

Let’s see this stack in action. Imagine you’ve just spun up Minikube and deployed Prometheus and Grafana using Helm. Your kubectl get pods -n monitoring might look something like this:

NAME                                     READY   STATUS    RESTARTS   AGE
prometheus-grafana-0                     2/2     Running   0          5m
prometheus-kube-state-metrics-0          1/1     Running   0          5m
prometheus-node-exporter-lqg4b           1/1     Running   0          5m
prometheus-prometheus-oper-prometheus-0  2/2     Running   0          5m

Here, prometheus-grafana-0 is your Grafana instance, prometheus-node-exporter-0 is collecting host metrics, prometheus-kube-state-metrics-0 is exposing Kubernetes object state as metrics, and prometheus-prometheus-oper-prometheus-0 is your Prometheus server.

The core problem this stack solves is the lack of visibility into your local Kubernetes cluster. Without it, debugging application issues, understanding resource utilization, or simply knowing if your deployments are healthy becomes a guessing game. Prometheus acts as the data collector and time-series database, scraping metrics from various exporters (like node-exporter for host-level metrics and kube-state-metrics for cluster-level object states) and your applications if they expose Prometheus-compatible endpoints. Grafana then queries Prometheus to visualize this data in dashboards.

The mental model for this stack is a data pipeline. Exporters are the data sources, Prometheus is the central ingestion point and storage, and Grafana is the presentation layer.

Here’s a typical Helm values.yaml for deploying Prometheus and Grafana to Minikube:

grafana:
  enabled: true
  adminPassword: "changeme"
  persistence:
    enabled: true
    size: 1Gi
  ingress:
    enabled: true
    hosts:
      - grafana.minikube.local

prometheus:
  prometheusSpec:
    serviceMonitorSelector: {} # Scrape all services
    podMonitorSelector: {}     # Scrape all pods
    storageSpec:
      volumeClaimTemplate:
        spec:
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 2Gi
    retention: 24h
    resources:
      requests:
        cpu: "200m"
        memory: "500Mi"
      limits:
        cpu: "500m"
        memory: "1Gi"

# kube-state-metrics and node-exporter are typically enabled by default
# but you can configure them here if needed.

To access Grafana, you’d typically port-forward: kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80. Then, navigate to http://localhost:3000 and log in with admin and the password you set (changeme in this example). You’ll need to configure Prometheus as a data source in Grafana, usually at http://prometheus-operated.monitoring.svc.cluster.local:9090.

A key lever you control is Prometheus’s scraping configuration. By default, the Prometheus Operator (which Helm usually installs) is configured to discover and scrape targets automatically using ServiceMonitor and PodMonitor custom resources. If your application isn’t showing up, ensure it has the correct labels that match the operator’s selectors, and that your application’s metrics endpoint is exposed. For example, a ServiceMonitor might look like this:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: my-app-monitor
  namespace: default
spec:
  selector:
    matchLabels:
      app: my-app # Label on your application's Service
  namespaceSelector:
    matchNames:
      - default
  endpoints:
  - port: metrics
    interval: 30s

The most overlooked aspect of this local setup is the resource allocation. Minikube, by default, might only allocate 2 CPUs and 4GB of RAM. Prometheus, especially with extensive scraping and retention, can easily consume a significant portion of this. If your Prometheus pod restarts with OOMKilled (Out Of Memory) or your Minikube VM becomes unresponsive, the first place to check is the resource requests and limits for Prometheus in your Helm values.yaml and the overall resources allocated to Minikube itself (e.g., minikube start --cpus 4 --memory 8192). The storageSpec for Prometheus is also critical; a small persistent volume will lead to Prometheus discarding old data too quickly or failing to store new data if it fills up.

Once you have basic dashboards working, the next natural step is to explore pre-built Grafana dashboards for Kubernetes, often found on Grafana Labs’ official dashboard repository, or to start crafting custom dashboards for your specific applications.

Want structured learning?

Take the full Minikube course →