K3s’s API server can be configured with custom flags, but it’s a bit of a hidden gem.
Let’s say you’re running K3s and you want to enable some specific feature or tune a particular aspect of the Kubernetes API server. Maybe you want to increase the rate limiting, or enable a beta API group that’s not on by default. Normally, you’d edit the API server’s manifest file in a full Kubernetes cluster. In K3s, it’s simpler, but you need to know where to look.
Consider this scenario: you’re developing a new Kubernetes operator, and you need to test an API that’s currently in beta and disabled by default in your K3s cluster. You’d normally have to edit the kube-apiserver manifest, but K3s abstracts this away.
Here’s how you’d enable the v1beta1 version of the apiextensions.k8s.io API group, which is often needed for Custom Resource Definitions (CRDs) that use older API versions.
First, locate your K3s configuration file. On most Linux systems, this will be /etc/rancher/k3s/config.yaml. If it doesn’t exist, you can create it.
You’ll need to add a kube-apiserver-arg section to this file. This tells K3s to pass specific arguments to the kube-apiserver binary when it starts.
kube-apiserver-arg:
- "enable-admission-plugins=...,ValidatingAdmissionWebhook,MutatingAdmissionWebhook"
- "runtime-config=apiextensions.k8s.io/v1beta1=true"
In this example, we’re doing two things:
- Ensuring
ValidatingAdmissionWebhookandMutatingAdmissionWebhookare enabled. These are often necessary for CRD controllers that use admission webhooks. The...indicates that any default plugins K3s might enable are preserved. - Crucially, we’re setting
runtime-config=apiextensions.k8s.io/v1beta1=true. This explicitly enables thev1beta1version of theapiextensions.k8s.ioAPI group.
After saving the config.yaml file, you need to restart the K3s service for the changes to take effect.
sudo systemctl restart k3s
You can then verify that the API is enabled by checking the API server’s configuration or by trying to create a CRD that uses v1beta1.
kubectl api-resources | grep apiextensions.k8s.io
You should see output that includes v1beta1 for customresourcedefinitions.
customresourcedefinitions crds apiextensions.k8s.io/v1beta1 true CustomResourceDefinition
customresourcedefinitions crds apiextensions.k8s.io/v1 true CustomResourceDefinition
This kube-apiserver-arg mechanism is incredibly powerful. You can use it to pass virtually any flag that the upstream Kubernetes kube-apiserver binary accepts. This includes things like:
--audit-log-path: To configure audit logging.--authorization-mode: To specify multiple authorization modes, likeNode,RBAC.--request-timeout: To set a global timeout for API requests.--service-cluster-ip-range: To define the IP address range for services.
The key is to know the available flags for the Kubernetes API server. The K3s documentation will list specific K3s-related configurations, but for general kube-apiserver flags, you’ll want to consult the official Kubernetes documentation for the kube-apiserver component.
The runtime-config flag is particularly useful for enabling or disabling specific API versions or features that are not enabled by default. It’s a string that takes the form group/version=true|false. For example, certificates.k8s.io/v1beta1=true would enable the v1beta1 version of the certificates.k8s.io API.
It’s important to remember that K3s aims for simplicity. While it exposes these advanced configuration options, it’s often the case that K3s has sensible defaults that cover most common use cases. You’ll typically only need to dive into kube-apiserver-arg when you’re running into specific edge cases or need to enable experimental features.
The list of available admission plugins can be found by running kube-apiserver --help on a standard Kubernetes installation, or by looking at the K3s source code if you want to be absolutely sure about what K3s exposes.
The next thing you’ll likely want to configure is the K3s controller manager, which has a similar mechanism for custom flags.