MLOps Infrastructure as Code: Terraform for ML Systems (2026)

Terraform can manage your ML infrastructure, but it’s not about writing AI models; it’s about defining the environment where they live and run.

Let’s see Terraform in action. Imagine you need to spin up a dedicated cloud environment for training a new deep learning model. This includes a virtual machine with specific GPU power, a secure network, and a place to store your datasets.

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "ml_trainer" {
  ami           = "ami-0c55b159cbfafe1f0" # Example: Deep Learning AMI (Ubuntu 20.04)
  instance_type = "g4dn.xlarge"         # GPU instance
  subnet_id     = aws_subnet.ml_subnet.id
  security_groups = [aws_security_group.ml_sg.id]

  tags = {
    Name = "ML_Trainer_Instance"
  }
}

resource "aws_vpc" "ml_vpc" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "ML_VPC"
  }
}

resource "aws_subnet" "ml_subnet" {
  vpc_id     = aws_vpc.ml_vpc.id
  cidr_block = "10.0.1.0/24"
  availability_zone = "us-east-1a"
  tags = {
    Name = "ML_Subnet"
  }
}

resource "aws_security_group" "ml_sg" {
  name        = "ml-security-group"
  description = "Allow SSH and specific ports for ML traffic"
  vpc_id      = aws_vpc.ml_vpc.id

  ingress {
    description = "SSH access"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"] # WARNING: Restrict this in production!
  }

  # Example: Allow traffic for a Jupyter Notebook server on port 8888
  ingress {
    description = "Jupyter Notebook"
    from_port   = 8888
    to_port     = 8888
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"] # WARNING: Restrict this in production!
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "ML_Security_Group"
  }
}

output "ml_trainer_public_ip" {
  description = "Public IP address of the ML training instance"
  value       = aws_instance.ml_trainer.public_ip
}

This Terraform configuration defines an AWS Virtual Private Cloud (VPC), a subnet within it, a security group to control network access, and finally, a GPU-enabled EC2 instance (g4dn.xlarge) that will host your ML workloads. The output block will reveal the public IP of the instance once it’s provisioned, allowing you to SSH into it.

The problem Terraform solves for MLOps is the reproducibility and consistency of your ML environments. Instead of manually clicking through cloud provider consoles, or relying on ad-hoc scripts, you have a declarative, version-controlled definition of your entire infrastructure. This means you can recreate the exact same training or deployment environment anywhere, anytime, reducing "it worked on my machine" syndrome and enabling seamless CI/CD for your ML pipelines.

Internally, Terraform operates on a declarative model. You tell it the desired state of your infrastructure (e.g., "I want one g4dn.xlarge instance in this VPC with these security rules"), and Terraform figures out the how to get there. It maintains a state file that tracks the actual resources it manages. When you run terraform apply, it compares your configuration to the current state and generates an execution plan detailing the changes needed. This plan is then executed to create, update, or destroy resources.

The levers you control are the resources you define (aws_instance, aws_vpc, aws_s3_bucket, google_compute_instance, etc.), their attributes (instance type, AMI ID, disk size, network configuration), and how they relate to each other (e.g., an instance being launched into a specific subnet). You can also manage data sources to pull in existing infrastructure or variables to parameterize your configurations, making them reusable across different projects or environments.

Many MLOps teams overlook the critical role of secrets management within their infrastructure code. While you can define resources like databases or storage buckets, sensitive information like API keys, database passwords, or TLS certificates should never be hardcoded. Instead, Terraform integrates with dedicated secrets management services (like AWS Secrets Manager, HashiCorp Vault, or GCP Secret Manager) using data sources. This ensures that your infrastructure can access necessary credentials without exposing them directly in your version control system, maintaining a robust security posture for your ML systems.

The next step is integrating this infrastructure definition into a CI/CD pipeline to automate provisioning and updates.