Skip to content

CKS: kubeadm config still uses kubeadm.k8s.io/v1beta3 — external-etcd clusters break on Kubernetes 1.37+ #14181

Description

@kiranchavala

problem

CKS writes a kubeadm configuration file into the control node's cloud-init, pinned to the v1beta3 API:

# plugins/integrations/kubernetes-service/src/main/resources/conf/k8s-control-node.yml:245
  - path: /etc/kubernetes/kubeadm-config.yaml
    ...
      apiVersion: kubeadm.k8s.io/v1beta3   # :249  (ClusterConfiguration)
      ...
      apiVersion: kubeadm.k8s.io/v1beta3   # :260  (InitConfiguration)

kubeadm.k8s.io/v1beta3 was deprecated when v1beta4 was introduced in Kubernetes 1.31, with support to be "removed after a minimum of 3 Kubernetes minor releases"
(Kubernetes v1.31: kubeadm v1beta4).
It has since been reported removed as of Kubernetes 1.37
(kubernetes-sigs/image-builder#2151).

Once removed, kubeadm init --config rejects the file and cluster creation fails.

Scope — this affects external-etcd clusters only. That config file is passed to kubeadm on exactly one path:

# k8s-control-node.yml:299-303
if [[ ${EXTERNAL_ETCD_NODES} == true ]]; then
  kubeadm init --config /etc/kubernetes/kubeadm-config.yaml --upload-certs
else
  kubeadm init --token {{ ... }} --token-ttl 0 {{ k8s_control_node.cluster.initargs }} --cri-socket /run/containerd/containerd.sock
fi

So:

Path Uses kubeadm config file? Affected on 1.37+
Control node, external etcd (:300) yes yes
Control node, stacked etcd (:302) no — flags only no
Additional control plane join (k8s-control-node-add.yml:239) no — flags only no
Worker join (k8s-node.yml:254) no — flags only no

grep -rn "kubeadm.k8s.io/v1beta" plugins/ systemvm/ returns only those two lines in the whole tree, so the fix is narrowly scoped.

Compounding this: CKS enforces no maximum supported Kubernetes version. addKubernetesSupportedVersion will happily accept 1.37.x, and the failure only surfaces part-way through cluster creation on the control node, which makes it hard to diagnose

versions

  • Affects main, 4.20, 4.21, 4.22 — the templated v1beta3 is identical on all of them.
  • Triggered by Kubernetes >= 1.37 (the release reported to have removed v1beta3).

The steps to reproduce the bug

  1. Build a CKS binaries ISO for Kubernetes 1.37.x (scripts/util/create-kubernetes-binaries-iso.sh).
  2. Register it: addKubernetesSupportedVersion semanticversion=1.37.0 url=...
  3. Create a Kubernetes cluster with external etcd nodes (etcdnodes=1 or more), which selects the
    kubeadm init --config path.
  4. On the control node, kubeadm init fails because /etc/kubernetes/kubeadm-config.yaml declares a
    config API version that kubeadm no longer recognises. Cluster creation does not complete.

A cluster on the same version without external etcd should succeed, since that path passes flags only —
which is a useful way to confirm the diagnosis.

logs

root@test-cks-etcd-control-1a0a97ce958:/opt/bin# ./deploy-kube-system status
error: your configuration file uses an old API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use kubeadm v1.36 instead and run 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version.
To see the stack trace of this error execute with --v=5 or higher
error: your configuration file uses an old API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use kubeadm v1.36 instead and run 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version.
To see the stack trace of this error execute with --v=5 or higher
error: your configuration file uses an old API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use kubeadm v1.36 instead and run 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version.
To see the stack trace of this error execute with --v=5 or higher
Error: kubeadm init failed!

What to do about it?

Migrate the template to kubeadm.k8s.io/v1beta4 — but not unconditionally. v1beta4 requires
kubeadm >= 1.31, and CKS still supports much older Kubernetes versions (there are comparisons against
1.15.0 and 1.16.0 in KubernetesClusterUpgradeWorker.java:75 and
KubernetesClusterService.MIN_KUBERNETES_VERSION_HA_SUPPORT), with versions registered by the operator.
A straight bump would break clusters on anything below 1.31.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions