K3s is a lightweight, certified Kubernetes distribution designed for edge computing, IoT, and CI/CD. When you install K3s, it typically comes with Traefik as the default ingress controller. However, you might want to switch to Nginx Ingress Controller for various reasons, such as familiarity, specific feature requirements, or performance tuning.
Here’s how to replace Traefik with Nginx Ingress Controller in your K3s cluster.
1. Uninstall Traefik
First, you need to remove the existing Traefik ingress controller. K3s installs Traefik as an add-on.
k3s-uninstall-traefik.sh
This script will remove the Traefik deployment, its associated Service, and any RBAC resources.
2. Install Nginx Ingress Controller
You can install the Nginx Ingress Controller using Helm, which is the recommended method for managing Kubernetes applications.
Add the ingress-nginx Helm repository:
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
Install the Nginx Ingress Controller. We’ll specify a nodeSelector to ensure it runs on a specific node if you have a multi-node cluster and want to control its placement. For a single-node setup, you can omit nodeSelector or select the appropriate node.
helm install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--set controller.nodeSelector.kubernetes\.io/hostname=your-node-hostname \
--set controller.replicaCount=2 \
--set controller.service.type=NodePort \
--set controller.admissionWebhooks.patch.nodeSelector."kubernetes\.io/hostname"=your-node-hostname
Explanation of key helm install arguments:
--namespace ingress-nginx --create-namespace: Creates a dedicated namespace for the ingress controller.--set controller.nodeSelector.kubernetes.io/hostname=your-node-hostname: This is crucial if you want to pin the ingress controller pods to specific nodes. Replaceyour-node-hostnamewith the actual hostname of your K3s node. You can find this by runningkubectl get nodes -o wide.--set controller.replicaCount=2: Ensures high availability by running two replicas of the ingress controller.--set controller.service.type=NodePort: Exposes the ingress controller via a NodePort. This is common in K3s/edge deployments where you might not have a cloud load balancer. TheNodePortwill be in the range30000-32767.--set controller.admissionWebhooks.patch.nodeSelector."kubernetes.io/hostname"=your-node-hostname: If you are using admission webhooks (which are enabled by default and recommended for features like automatic certificate generation with cert-manager), you need to ensure the webhook pod also runs on the selected node(s).
After installation, verify the Nginx Ingress Controller pods are running:
kubectl get pods -n ingress-nginx
You should see pods like ingress-nginx-controller-... in a Running state.
3. Configure DNS
The Nginx Ingress Controller will now be listening on the NodePort you exposed. You’ll need to configure your DNS to point your desired domain names to the IP addresses of your K3s nodes on that specific NodePort.
For example, if your K3s node has IP 192.168.1.100 and the NodePort for the ingress controller service is 31234, you would create DNS A records like:
myapp.yourdomain.com -> 192.168.1.100api.yourdomain.com -> 192.168.1.100
Alternatively, you can set up a load balancer (like HAProxy, MetalLB for bare-metal, or a cloud provider LB) in front of your K3s nodes to distribute traffic and provide a stable IP address. If using MetalLB, you would configure its LoadBalancer service type.
4. Create an Ingress Resource
Now, you can create an Ingress resource to route traffic to your services.
Let’s say you have a Deployment and Service for a simple web application named my-web-app in the default namespace.
Deployment (e.g., my-web-app-deployment.yaml):
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-web-app
spec:
replicas: 2
selector:
matchLabels:
app: my-web-app
template:
metadata:
labels:
app: my-web-app
spec:
containers:
- name: web
image: nginxdemos/hello:plain-text
ports:
- containerPort: 80
Service (e.g., my-web-app-service.yaml):
apiVersion: v1
kind: Service
metadata:
name: my-web-app-svc
spec:
selector:
app: my-web-app
ports:
- protocol: TCP
port: 80
targetPort: 80
Apply these:
kubectl apply -f my-web-app-deployment.yaml
kubectl apply -f my-web-app-service.yaml
Ingress Resource (e.g., my-web-app-ingress.yaml):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-web-app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx # This is important!
rules:
- host: myapp.yourdomain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-web-app-svc
port:
number: 80
Apply the Ingress resource:
kubectl apply -f my-web-app-ingress.yaml
Key points in the Ingress resource:
ingressClassName: nginx: This tells Kubernetes to use the Nginx Ingress Controller for this Ingress resource. Make sure this matches theingressClassNameconfigured by your Helm chart (which defaults tonginx).host: myapp.yourdomain.com: The domain name that will be used to access your application. This must match your DNS configuration.path: /,pathType: Prefix: Routes all requests to the root path.backend.service.nameandbackend.service.port.number: Specifies which Service and port to forward traffic to.nginx.ingress.kubernetes.io/rewrite-target: /: This is a common annotation for Nginx Ingress. It rewrites the URL path to/before forwarding it to the backend service. This is often needed when your application expects requests directly at its root, but the Ingress path might be more specific.
Once DNS is propagated and the Ingress is applied, you should be able to access http://myapp.yourdomain.com (or http://<node-ip>:<node-port> if testing directly without DNS) and see the output from the nginxdemos/hello application.
You have now successfully replaced Traefik with Nginx Ingress Controller in your K3s cluster.
The next challenge will be managing TLS certificates for HTTPS traffic.