Run ML Workloads on GKE TPU Node Pools (2026)

TPU node pools in GKE let you run machine learning workloads directly on Google’s custom AI accelerators.

Let’s see it in action. Imagine you’ve got a TensorFlow model you want to train. You’d start by creating a GKE cluster, but with a specific node pool configuration for TPUs.

apiVersion: container.googleapis.com/v1
kind: Cluster
metadata:
  name: tpu-cluster
spec:
  initialNodeCount: 1
  nodePools:
  - name: default-pool
    initialNodeCount: 1
  - name: tpu-pool
    nodeConfig:
      machineType: n1-standard-8
      acceleratorConfig:
        count: 8
        type: v3-8
    autoscaling:
      minNodeCount: 0
      maxNodeCount: 4
    management:
      autoRepair: true
      autoUpgrade: true

This configuration defines a cluster named tpu-cluster. It has a default-pool for general workloads and a tpu-pool specifically for TPUs. The tpu-pool uses n1-standard-8 machines, each with 8 v3 TPUs (v3-8). It’s also set up to autoscale between 0 and 4 such nodes.

Once your cluster is ready, you deploy your ML application. This typically involves a Kubernetes Deployment that specifies a container image. Inside this container, your ML framework (like TensorFlow or PyTorch) will automatically detect and utilize the TPUs.

Here’s how a simple TensorFlow job might look:

apiVersion: batch/v1
kind: Job
metadata:
  name: tpu-training-job
spec:
  template:
    spec:
      containers:
      - name: trainer
        image: gcr.io/cloud-ai-platform/tf_containers:latest-tpu
        command: ["python", "/app/train.py"] # Your training script
        resources:
          limits:
            google.com/tpu: 8 # Request 8 TPUs
      restartPolicy: Never

This Job will run a container from a pre-built TensorFlow TPU image. The resources.limits.google.com/tpu: 8 is crucial; it tells Kubernetes to schedule this pod only on nodes that can provide 8 TPUs, which in our case is a node from tpu-pool. Your train.py script would then be written to leverage tf.distribute.TPUStrategy.

The core problem this solves is the high cost and complexity of managing dedicated TPU hardware. Instead of buying and maintaining physical TPUs, you get them as managed, scalable resources within GKE. The system handles the underlying infrastructure, device allocation, and driver management, allowing you to focus on your model.

Internally, GKE integrates with Google Cloud TPU resources. When you create a TPU node pool, GKE provisions the necessary TPU VMs and attaches them to your cluster. The Kubernetes scheduler, with the help of device plugins, understands the google.com/tpu resource and directs pods requesting TPUs to these specialized nodes. For frameworks like TensorFlow, specific libraries and APIs are designed to communicate with the TPUs through high-speed interconnects.

A key detail is how TPU versions map to GKE node configurations. For example, v3-8 implies 8 TPU cores of type v3, often configured as a single TPU device for distributed training. When you request google.com/tpu: 8 in your pod spec, you’re essentially asking for a slice of that hardware that can be utilized by your job. The acceleratorConfig.type in the node pool definition dictates the type and size of the TPU device available on that node. If you request more TPUs than a single node provides (e.g., google.com/tpu: 16 on a v3-8 node pool), Kubernetes will try to schedule your pod across multiple nodes in that pool.

The next step is optimizing your distributed training strategy across multiple nodes.