K3s can’t pull images from a private registry by default because it doesn’t have any credentials to authenticate with it.
Let’s see this in action. Imagine you have a private registry at registry.example.com and an image my-app:v1.0. If you try to deploy this to K3s:
apiVersion: v1
kind: Pod
metadata:
name: my-app-pod
spec:
containers:
- name: my-app-container
image: registry.example.com/my-app:v1.0
K3s will try to pull registry.example.com/my-app:v1.0 and fail with a ErrImagePull or ImagePullBackOff status. The underlying Kubernetes component, kubelet, is responsible for pulling images. When kubelet tries to pull an image from a private registry and encounters an authentication challenge, it needs a way to provide those credentials.
The standard Kubernetes way to handle this is with imagePullSecrets. These are Kubernetes Secret objects of type kubernetes.io/dockerconfigjson that contain your registry credentials.
Here’s how you set it up:
First, create a Docker config.json file that contains your registry credentials. This file typically looks like this:
{
"auths": {
"registry.example.com": {
"auth": "dXNlcm5hbWU6cGFzc3dvcmQ="
}
}
}
The auth field is a Base64 encoded string of username:password. You can generate this using echo -n 'your_username:your_password' | base64. For example, if your username is admin and password is supersecret, it would be echo -n 'admin:supersecret' | base64, resulting in YWRtaW46c3VwZXJzZWNyZXQ=.
Now, create a Kubernetes Secret from this file. On a system with kubectl configured for your K3s cluster, you’d run:
kubectl create secret docker-registry my-registry-secret \
--docker-server=registry.example.com \
--docker-username=admin \
--docker-password=supersecret \
--docker-email=admin@example.com \
--namespace=default
This command directly creates the Secret object with the correct type and content. The --docker-email is optional but often required by registries.
Alternatively, if you have the config.json file already prepared, you can create the secret like this:
kubectl create secret generic my-registry-secret \
--from-file=.dockerconfigjson=/path/to/your/config.json \
--type=kubernetes.io/dockerconfigjson \
--namespace=default
Once the secret is created, you need to tell your Pod (or Deployment, StatefulSet, etc.) to use it. You do this by adding an imagePullSecrets field to the Pod’s spec:
apiVersion: v1
kind: Pod
metadata:
name: my-app-pod
spec:
containers:
- name: my-app-container
image: registry.example.com/my-app:v1.0
imagePullSecrets:
- name: my-registry-secret
When kubelet tries to pull registry.example.com/my-app:v1.0, it will look for imagePullSecrets in the Pod’s namespace. It finds my-registry-secret, decodes the credentials from the Secret, and uses them to authenticate with registry.example.com.
The imagePullSecrets mechanism is generic. It works for any registry that supports basic authentication or token-based authentication that can be represented in the Docker configuration format. This includes Docker Hub, Quay.io, Harbor, Nexus, and many others. The key is that the credentials must be in the ~/.docker/config.json format that Kubernetes expects.
If you’re using K3s on multiple nodes, you need to ensure this my-registry-secret exists in every namespace where you intend to run pods that need to pull from your private registry. You can also create it in the kube-system namespace if you want all system components to have access, though this is less common.
A common pitfall is forgetting to specify the namespace when creating the secret or when referencing it in the Pod spec. Secrets are namespace-scoped.
The next thing you’ll likely run into is managing TLS certificates for your private registry if it’s not using a publicly trusted CA.