The Istio control plane failed to initialize because the Kubernetes API server for one of your clusters is unreachable from the cluster where you’re trying to create the Istio control plane.

The most common culprit is network connectivity. Your Istio control plane pod (usually istiod) needs to talk to the Kubernetes API server of the remote cluster to manage its resources. If that connection is blocked, Istio can’t function.

Cause 1: Firewall Rules Blocking Egress Traffic

Your cloud provider’s firewall or an on-premises firewall is preventing the Istio control plane pods from initiating outbound connections to the remote cluster’s Kubernetes API server.

  • Diagnosis: From a pod within your Istio control plane’s namespace (e.g., istio-system), try to curl the remote cluster’s API server endpoint. You’ll need the FQDN and port of the remote cluster’s API server, which you can get from its kubeconfig file. For example, if your remote API server is at https://192.168.1.100:6443:
    kubectl exec -n istio-system <istiod-pod-name> -- curl -v https://<remote-api-server-address>:<port>
    
    If this times out or returns a connection refused error, it’s likely a firewall.
  • Fix: Update your network security groups or firewall rules to allow egress TCP traffic on port 6443 (or your cluster’s API server port) from the IP range of your Istio control plane’s nodes to the IP address of the remote cluster’s API server. For AWS Security Groups, this would look like:
    Type: Custom TCP
    Protocol: TCP
    Port Range: 6443
    Source: <IP Range of Istio Control Plane Nodes>
    
    For GCP Firewall rules:
    Direction: Egress
    Protocols and ports: tcp:6443
    Destination IP ranges: <IP Address of Remote API Server>
    Source IP ranges: <IP Range of Istio Control Plane Nodes>
    
  • Why it works: This explicitly permits the necessary network path for istiod to communicate with the remote Kubernetes API.

Cause 2: Incorrect kubeconfig for Remote Cluster Context

When setting up multicluster, you typically provide kubeconfig files for each cluster. If the kubeconfig for the remote cluster is malformed, outdated, or points to the wrong API server address, Istio won’t be able to connect.

  • Diagnosis: Use kubectl config view --kubeconfig=<path-to-remote-kubeconfig> to inspect the remote cluster’s configuration. Pay close attention to the server: field under the clusters: section. Verify this address is correct and reachable from where Istio is running. Then, try to use this kubeconfig to access the remote cluster:
    kubectl --kubeconfig=<path-to-remote-kubeconfig> get nodes
    
    If this command fails with an authentication or connection error, your kubeconfig is the problem.
  • Fix: Obtain the correct kubeconfig file for your remote cluster. This usually involves downloading it from your cloud provider’s console or generating it from your cluster’s administration interface. Ensure the server: address within the kubeconfig accurately reflects the remote cluster’s API endpoint. If you’re using the Istio operator or istioctl, you’ll need to re-apply the configuration with the corrected kubeconfig reference. For example, when using the Istio operator, update the remote: section of your IstioOperator CR:
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    metadata:
      name: istio-multicluster
    spec:
      profile: default
      multicluster:
        clusterName: cluster2
        clusterConfig:
          network: network2
        # The Istio operator needs access to the remote cluster's API.
        # This can be provided via a secret containing the kubeconfig.
        # The secret must be in the same namespace as the Istio operator.
        remoteKubeconfigSecret: cluster2-kubeconfig
    
    Then create/update the secret cluster2-kubeconfig with the valid kubeconfig.
  • Why it works: A valid kubeconfig provides istiod with the correct endpoint and credentials to authenticate and connect to the remote Kubernetes API server.

Cause 3: Network Policy Blocking Pod-to-Pod Communication

Kubernetes NetworkPolicy resources might be in place that prevent pods in the istio-system namespace from establishing outbound connections to the Kubernetes API server’s IP address and port.

  • Diagnosis: Check for any NetworkPolicy resources in the istio-system namespace and any namespaces that might be applied to the nodes where your Istio control plane pods are running.
    kubectl get networkpolicy -n istio-system
    kubectl get networkpolicy --all-namespaces # to find policies on other namespaces/nodes
    
    If policies exist, examine their egress rules. A restrictive policy might be blocking the necessary outbound traffic.
  • Fix: Modify the relevant NetworkPolicy to allow egress traffic from istio-system pods to the IP address and port of the remote cluster’s API server. For example, to allow egress to a specific IP:
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: allow-istiod-egress-to-remote-api
      namespace: istio-system
    spec:
      podSelector:
        matchLabels:
          app: istiod # or whatever label matches your istiod pod
      policyTypes:
      - Egress
      egress:
      - to:
        - ipBlock:
            cidr: <remote-api-server-ip>/32 # e.g., 192.168.1.100/32
        ports:
        - protocol: TCP
          port: 6443
    
  • Why it works: This explicitly grants permission for istiod to send traffic to the remote API server, bypassing restrictive network policies.

Cause 4: DNS Resolution Failure for Remote API Server

The Istio control plane pods cannot resolve the hostname of the remote cluster’s API server. This can happen if the DNS configuration within the control plane cluster is not set up to resolve external or cluster-specific DNS names.

  • Diagnosis: Exec into an istiod pod and try to resolve the remote API server’s hostname using nslookup or dig.
    kubectl exec -n istio-system <istiod-pod-name> -- nslookup <remote-api-server-hostname>
    
    If this command fails to return an IP address, DNS resolution is the issue.
  • Fix: Ensure that your Kubernetes cluster’s DNS (e.g., CoreDNS) is configured to properly resolve the hostname of the remote API server. This might involve configuring upstream DNS servers or adding specific forward or rewrite rules in your CoreDNS configuration if the remote API server is on a private network. If using istioctl install, you can specify custom DNS settings:
    istioctl install --set profile=default \
      --set values.global.proxy.clusterDomain=cluster.local \
      --set values.pilot.env.EXTERNAL_ISTIOD=true \
      --set values.pilot.env.MESH_NETWORKING_MODE=MULTI \
      --set values.global.meshNetworks='{
        network1: { endpoints: [{ fromRegistry: REGISTRY1 }] },
        network2: { endpoints: [{ fromRegistry: REGISTRY2 }] }
      }' \
      --set values.istiod.enableCRDT=true \
      --set values.meshConfig.accessLogFile=/dev/stdout \
      --set values.pilot.env.RESOURCENAMESPACE=istio-system \
      -f cluster1-config.yaml \
      -f cluster2-config.yaml
    
    And ensure your cluster2-config.yaml (or equivalent) has the correct server address and that your cluster’s DNS can resolve it. If your cluster DNS can’t resolve it, you might need to manually add an entry to /etc/hosts within the istiod pod or configure CoreDNS.
  • Why it works: Correct DNS resolution allows istiod to translate the API server’s hostname into an IP address it can use to establish a connection.

Cause 5: TLS Certificate Issues for Remote API Server

The TLS certificate presented by the remote Kubernetes API server is not trusted by the Istio control plane pods. This can happen if the certificate is self-signed and not added to the trust store of the istiod pods, or if there are intermediate certificate issues.

  • Diagnosis: The curl command from Cause 1, when successful in reaching the API server but failing due to TLS, will show specific certificate errors (e.g., "certificate verify failed," "unable to get local issuer certificate").
  • Fix: Ensure that the CA certificate that signed the remote API server’s certificate is trusted by the Istio control plane nodes. This often involves adding the CA certificate to the system’s trust store on the nodes or mounting it into the istiod pods. If you are using a managed Kubernetes service, ensure you are using the correct CA bundle provided by the service. If it’s a custom setup, you might need to explicitly provide the CA bundle to istiod. For example, if you’re using istioctl install and need to provide a custom CA bundle:
    istioctl install --set profile=default \
      -f istio-operator.yaml \
      --set values.pilot.caCertificates='[{certFile: /path/to/your/ca.crt, keyFile: /path/to/your/ca.key}]'
    
    You would then need to ensure ca.crt is accessible to the istiod pods, possibly via a ConfigMap mounted as a volume.
  • Why it works: By trusting the CA that issued the remote API server’s certificate, istiod can successfully validate the identity of the server and establish a secure TLS connection.

Cause 6: Resource Constraints on the Control Plane Cluster

The cluster hosting the Istio control plane might be under heavy load or experiencing resource exhaustion (CPU, memory), preventing istiod from starting up correctly or making outbound calls.

  • Diagnosis: Check the resource utilization of the nodes in your control plane cluster. Monitor CPU and memory usage for the istio-system namespace and specifically for the istiod pods.
    kubectl top nodes
    kubectl top pods -n istio-system
    kubectl describe pod <istiod-pod-name> -n istio-system
    
    Look for high CPU/memory usage or istiod pods that are in a CrashLoopBackOff or Pending state due to resource issues.
  • Fix: Increase the resources allocated to the nodes in your control plane cluster or reduce the load on the cluster. This might involve scaling up the node instance types, adding more nodes, or optimizing other workloads running on the cluster. You can also adjust the resource requests and limits for the istiod deployment within the Istio operator configuration:
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    metadata:
      name: istio-control-plane
    spec:
      profile: default
      components:
        pilot:
          k8s:
            resources:
              requests:
                cpu: 500m
                memory: 1Gi
              limits:
                cpu: 1000m
                memory: 2Gi
    
  • Why it works: Providing adequate resources ensures that istiod can start, run, and perform its network operations without being throttled or terminated by the operating system or Kubernetes.

The next error you’ll likely encounter if you fix this is related to the service discovery or configuration synchronization between the clusters, potentially manifesting as STALE_EDGES or issues with traffic routing not behaving as expected.

Want structured learning?

Take the full Istio course →