K3s is designed to be lightweight, but "lightweight" doesn’t mean "insecure."
Here’s how to take a fresh K3s install and make it production-ready, focusing on hardening and secure deployment.
First, let’s get a basic K3s server up and running. On your chosen server, run:
curl -sfL https://get.k3s.io | sh -
This will install K3s with default settings. Now, let’s secure it.
Secure Access to the Kubernetes API
By default, kubectl can access the K3s API from the server itself without authentication. This is a big no-no for production.
Diagnosis: Check the kubeconfig file.
sudo cat /etc/rancher/k3s/k3s.yaml
You’ll see server: https://127.0.0.1:6443. For remote access, you need to bind to a public IP and enable TLS.
Cause 1: Default API Server Binding
K3s binds the API server to 127.0.0.1 by default, meaning it’s only accessible locally.
Fix: Reconfigure K3s to bind to a specific IP address and enable TLS.
Edit /etc/rancher/k3s/config.yaml (or create it if it doesn’t exist) and add:
tls-san:
- your_server_public_ip
- your_server_hostname
write-kubeconfig-mode: "0644"
Then, restart K3s:
sudo systemctl restart k3s
This tells K3s to generate certificates that include your specified IP and hostname, allowing remote clients to connect securely. write-kubeconfig-mode ensures the kubeconfig file is readable by the user who needs it.
Cause 2: Missing Authentication/Authorization Even with TLS, if you don’t configure authentication and authorization properly, anyone who can reach the API server could potentially do anything.
Fix: K3s enables TLS authentication by default using client certificates. The kubeconfig file generated at /etc/rancher/k3s/k3s.yaml already contains a client certificate for system:admin. To use this remotely, you need to copy this file to your local machine and update the server address to your server’s public IP.
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ...
server: https://your_server_public_ip:6443 # <-- Change this
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
client-certificate-data: ...
client-key-data: ...
Replace your_server_public_ip with the actual IP. Now kubectl --kubeconfig /path/to/your/remote.yaml get nodes should work.
Network Policies for Pod-to-Pod Security
By default, all pods in a K3s cluster can communicate with each other. This is a significant security risk.
Diagnosis: Deploy a simple application and observe its network access.
apiVersion: v1
kind: Pod
metadata:
name: busybox-test
labels:
app: net-test
spec:
containers:
- name: busybox
image: busybox
command: ["sleep", "3600"]
kubectl apply -f pod.yaml
kubectl exec busybox-test -- wget -qO- http://other-pod-ip:port
Without network policies, this will succeed for any other pod.
Cause 3: CNI Default Behavior (Flannel) K3s uses Flannel as its default CNI. Flannel, by default, allows all pod-to-pod communication.
Fix: Implement Kubernetes Network Policies. You need to install a CNI that supports NetworkPolicy, like Calico or Cilium. K3s can be installed with a different CNI. For a new install:
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--flannel-backend=none" sh -
# Then install Calico manually or use the K3s install script with Calico
# For example, using K3s install with Calico:
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik --disable local-storage --token mysecrettoken --cluster-cidr 10.42.0.0/16 --service-cidr 10.43.0.0/16 --cni calico" sh -
Once Calico is installed (or another NetworkPolicy-compliant CNI), you can define policies.
Example: Deny all ingress by default for pods with label app=backend.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-all-ingress
namespace: default
spec:
podSelector:
matchLabels:
app: backend
policyTypes:
- Ingress
This policy, when applied, will block all incoming traffic to pods labeled app=backend unless explicitly allowed by another policy.
Secure Secrets Management
Storing sensitive data like database passwords directly in Kubernetes Secrets is better than plain text, but K3s’s default Secret encryption is not enabled.
Diagnosis: Create a secret and inspect its contents.
kubectl create secret generic my-secret --from-literal=password=mypassword
kubectl get secret my-secret -o yaml
You’ll see data is base64 encoded, not encrypted at rest.
Cause 4: Etcd Encryption Not Enabled
K3s uses SQLite by default, and its etcd (if enabled) or internal datastore doesn’t encrypt secrets at rest without explicit configuration.
Fix: Enable Encryption at Rest for Etcd.
If you’re using K3s with external etcd, you need to configure it. For K3s’s embedded etcd (which is the default when not using external etcd), you can enable encryption.
During installation or via config.yaml:
# For new installs
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--secret-encryption-key <your-32-byte-key>" sh -
# For existing installs, edit /etc/rancher/k3s/config.yaml
# Add:
# secret-encryption-key: <your-32-byte-key>
# Then restart K3s:
# sudo systemctl restart k3s
Generate a 32-byte key:
openssl rand -base64 32
This key is used to encrypt sensitive data like Secrets when they are stored in etcd. You must not lose this key, or you will be unable to decrypt your secrets.
Limiting Default Privileges
Pods running with excessive privileges can be a major security hole.
Diagnosis: Check the default service account or deploy a pod with elevated privileges.
A default ServiceAccount in Kubernetes has broad permissions.
Cause 5: Default Service Account Permissions
The default service account in the kube-system namespace has permissions to list and watch pods, nodes, and services.
Fix: Use dedicated Service Accounts with minimal privileges and restrict default permissions.
-
Disable auto-mounting of the Service Account token: In your Pod spec or Deployment/StatefulSet template, add:
spec: serviceAccountName: my-app-sa automountServiceAccountToken: falseThis prevents pods from automatically getting a Service Account token if they don’t explicitly need one.
-
Create Role-Based Access Control (RBAC) for specific Service Accounts: Define
RoleorClusterRoleandRoleBindingorClusterRoleBindingto grant only the necessary permissions to your application’s Service Account. Example: A Service Account that can only read Pods in its own namespace.apiVersion: v1 kind: ServiceAccount metadata: name: read-pods-sa namespace: default --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: pod-reader namespace: default rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: read-pods-binding namespace: default subjects: - kind: ServiceAccount name: read-pods-sa namespace: default roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io
Regularly Update K3s and Kubernetes Components
Vulnerabilities are discovered regularly. Staying up-to-date is critical.
Diagnosis: Check your current K3s version.
k3s --version
Cause 6: Outdated Software Running an older version of K3s means you’re missing security patches and bug fixes.
Fix: Follow the official K3s upgrade process. For a typical installation, this involves running the K3s installer script again with the desired version.
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.27.10+k3s1" sh -
Always back up your etcd data (if using external etcd) or your K3s state directory (/var/lib/rancher/k3s/server/db/) before upgrading.
After these steps, your next immediate concern will likely be securing ingress traffic to your services and managing TLS certificates for them.