feat(kubernetes): boot tenant worker disks from a shared golden Talos image (CDI clone) - #3294

Code Review

This pull request introduces the kubernetes-worker-image package to pre-populate golden Talos worker OS images as DataVolumes in the cozy-public namespace. This enables tenant Kubernetes worker VMs to boot via CDI storage-layer clones instead of downloading raw images over HTTP, resolving node-join flakes and eliminating external dependencies during worker provisioning. The kubernetes app is updated to utilize these clones while maintaining backward compatibility for existing node groups. Feedback on the changes identifies that the fallback storage size in dv.yaml is set to 4Gi, which is smaller than the ~4.15 GiB Talos raw image and would cause failures. It is recommended to increase this fallback to 6Gi and quote the rendered value.

NOT LGTM — the golden-image package is wired opt-in (off by default), so neither the production import reduction nor the e2e node-join flake fix this PR is premised on actually happens; the image-cache workaround is removed in the same change, leaving e2e more exposed than before.

Business context: Replace per-worker HTTP import of the Talos OS image from the public Image Factory with a per-worker CDI clone of a shared golden image in cozy-public, to remove the per-worker Factory dependency and the kubernetes-* e2e node-join flake (#3231).

Blockers

B1: The golden package is opt-in (default OFF), so the feature is inert by default and the description/release-note are inverted

File: packages/core/platform/templates/bundles/iaas.yaml:123

Issue: kubernetes-worker-image is registered via cozystack.platform.package.optional.default. That helper emits the Package only when the name is in bundles.enabledPackages (_helpers.tpl: if and (has $name $enabled) (not (has $name $disabled))). It is opt-in / disabled-by-default, not opt-out.

Evidence: the optional helper body gates on has $name $enabled. The sibling vm-default-images uses the same helper and its own changelog (v1.3.4 / v1.4.1) documents it as "disabled by default … enable via bundles.enabledPackages". No default enabledPackages in packages/core/platform/values.yaml lists either package. So on a stock install the golden DataVolume is never created and every worker takes the HTTP-import fallback.

Impact: the production benefit ("one import per image instead of one per worker") does not occur out of the box. The release note ("New node groups adopt the clone automatically") is false by default — with no golden, nothing clones. The description's "Opt-out via bundles.disabledPackages" is inverted.

Fix: decide the intent. If the golden should be active by default, use cozystack.platform.package.default (always-on). If opt-in is intended (a 6Gi replicated volume on every install, including VM-only / app-only clusters, is a real cost — the reason vm-default-images is opt-in), correct the description and release note to "opt-in via enabledPackages" and address B2.

B2: e2e never enables the golden, and the image-cache it removes was the only #3231 mitigation there

File: hack/e2e-install-cozystack.bats:127

Issue: the e2e installer sets bundles.enabledPackages: [cozystack.external-dns-application] only — kubernetes-worker-image is absent. Combined with B1, the golden is not deployed in e2e, so kubernetes-latest / kubernetes-previous workers hit $useClone=false and HTTP-import from the default https://factory.talos.dev. This PR also removes the in-sandbox talos-image-cache mirror that buffered that public endpoint.

Evidence: run-kubernetes.sh no longer injects talos.imageFactoryURL; no chainsaw fixture sets it; the new diagnostics comment itself says "a cluster that fell back to the HTTP import instead shows an importer-* pod looping on a factory error". The green E2E run is not proof — #3231 is by definition an intermittent factory stall, so one pass is consistent with "the factory happened to be reachable".

Impact: the PR's headline justification (fix the e2e flake) is not delivered, and e2e reliability likely regresses versus main (buffer removed, nothing replacing it).

Fix: add cozystack.kubernetes-worker-image to the e2e enabledPackages (the default golden schematicID/version already match packages/apps/kubernetes talos ce4c98… / v1.13.6, so the clone would be selected), and confirm a worker-spinning suite actually takes the clone path before removing the cache.

B3: New clone-selection branching ships with zero unit coverage in a chart that already mocks lookup

File: packages/apps/kubernetes/templates/cluster.yaml:110-127

Issue: the live-KMT source-type pin (http stays http, pvc stays pvc) and the new-group clone-adoption are the safety-critical "no worker roll" guarantee, yet no unit test exercises any of the new branches — the existing tests only cover the lookup-nil HTTP path.

Evidence: packages/apps/kubernetes/tests/nodegroups_default_test.yaml already uses kubernetesProvider.objects to mock lookup for MachineDeployment and KubevirtMachineTemplate — the exact kinds this logic reads. The pin path (highest blast radius: breaking byte-identity rolls every worker in every tenant cluster on the next platform bump) is covered by neither unit tests nor e2e (e2e is fresh-install only, so every group is new).

Fix: add helm-unittest cases with mocked lookups: (a) live MD + KMT with http disk-system renders http and keeps usePopulator: "false"; (b) live MD + KMT with pvc renders the clone; (c) no live MD plus a golden DataVolume present renders the clone.

Non-blocking follow-ups

1. packages/system/kubernetes-worker-image/templates/dv.yaml:32 — the | default "4Gi" tail contradicts the documented ~4.15Gi minimum; if $.Values.storage is ever cleared it renders a too-small PVC (DataVolume too small). Make the final fallback 6Gi and quote it (already flagged by the Gemini bot).
2. packages/apps/kubernetes/templates/cluster.yaml:127 — $useClone is latched on golden existence, not readiness. A new group rendered during the golden's import window clones an unpopulated source with no HTTP fallback; the worker DataVolume blocks until the golden reaches Succeeded, or indefinitely if the golden import fails. Consider gating on dig "status" "phase" "" $goldenDV | eq "Succeeded".
3. Cross-StorageClass clone: if a tenant's worker storageClass differs from the golden's (replicated), CDI cannot CSI-clone and silently falls back to a host-assisted network copy — documented in values.yaml but unguarded. Fine to leave, worth a note.

Verdict

NOT LGTM

feat(kubernetes): boot tenant worker disks from a shared golden Talos image (CDI clone). Two MAJOR issues around catalog-entry lifecycle and version-drift guarding block it. Verified by execution: helm-unittest across three charts, go build/go vet, and a render-diff of the default path against main (byte-identical apart from random tokens).

Findings

[MAJOR] packages/system/kubernetes-worker-image/templates/dv.yaml:6-32 + packages/apps/kubernetes/templates/cluster.yaml:142-145 — removing a catalog images[] entry silently blocks ALL future reconciliation of any existing tenant pinned to it. The golden DataVolume lacks helm.sh/resource-policy: keep, so removing (rather than appending) an entry prunes the DataVolume + its CDI PVC. Then cluster.yaml:142 does a lookup on that golden and cluster.yaml:144 fail()s the entire chart render when nil — aborting the whole Kubernetes HelmRelease, not just the disk block. An unrelated tenant then gets stuck on its next Helm action (scale / addon / RBAC edit). This is strictly worse than the vm-disk precedent, which fails at resource level only. Add resource-policy: keep to the golden DV and/or make the missing-golden path degrade instead of failing the whole render.

[MAJOR] packages/apps/kubernetes/values.yaml:307-308 vs packages/system/kubernetes-worker-image/values.yaml:44-46 — no automated guard ties the tenant chart's default Talos (version, schematicID) to the catalog's default images[] entry. They are two independently-edited files; a future one-sided bump silently breaks every fresh install using image.builtin: {}, and no existing test catches the drift. Add a test/lint asserting the two defaults stay in sync.

Claim mismatches / caveats

• [PARTIAL] Stale unit-test counts in the PR body; actual is 193/193 kubernetes, 88/88 platform, all green.
• [UNVERIFIABLE] live-cluster claim — outside the hermetic toolset.

Reconciliation with prior feedback (lexfrei CHANGES_REQUESTED)

• These findings are on different grounds than lexfrei's B1 — independent convergence, not overlap.
• lexfrei's B1 "image-cache workaround removed leaving e2e more exposed" is addressed: the removed talos-image-cache.sh / e2e-talos-image-cache.yaml / talos-image-cache_test.bats are replaced by the golden clone (hack/e2e-chainsaw/_lib/run-kubernetes.sh:334-339 sets image.builtin: {} on worker groups and waits for the golden import at 263-269, explicitly as the #3231 mitigation).
• lexfrei's "opt-in / off by default, so the production import-reduction premise doesn't happen" is not resolved for the production default path (by design): the default/omitted path renders identically to main and does not use the golden clone. The feature was deliberately reframed as an opt-in catalog, so production imports are only reduced for tenants who explicitly opt in.

Verdict

NOT LGTM

First, the size. 6555 changed lines across 37 files is past what one review round can hold, and roughly 2400 of those deletions are the ghcr.io pull-through mirror teardown that belongs to the stacked #4007, not to this feature. I reviewed the golden-image change and looked at the mirror removal only where the two touch. The natural seam is exactly that split: land #4007 on its own, and this PR shrinks to the catalog package, the kubernetes-nodes render path, and the schema. What follows is the feature's review, not a full account of the mirror teardown.

On the feature itself: the clone path holds up under render diff and per-conjunct mutation, and adopting it rolls no existing worker. Four things block merge. The catalog's operator knobs cannot be set through any supported path, so the two scenarios the PR body opens with (air-gapped and non-replicated storage) have no route to the fix the guards tell operators to apply. The extra transient pool capacity the clone needs is written down only in the CI sandbox. The new package's own instructions point an operator at a Kubernetes CR field that has been inert since the Phase 2 split. And talos.version's description promises a support-matrix guarantee that the guard's own contract, in this same PR, says it does not provide.

Still open from my earlier rounds

Open, second round: catalog value changes collide with DataVolume spec immutability (packages/system/kubernetes-worker-image/templates/dv.yaml:39, raised 2026-08-19). The golden's name is still keyed only on (schematicID, version) at dv.yaml:30, while storage, storageClass and the source URL stay ordinary mutable spec fields with no render-time guard against a bump on an entry whose golden exists. What changed is the documentation, not the code: values.yaml:62 now says an entry is "effectively immutable once its golden exists" and values.yaml:76 spells out that raising storage "is rejected on the next upgrade... the only way out is deleting the golden". Documenting a footgun is worth something, but the recovery it names walks straight into the deletion blast radius under Operational risks below. A render-time lookup on the existing golden that fails with "entry X changed after its golden was created, add a new entry instead" costs about as much as the two paragraphs already written.

Partially closed: the support matrix now has a second call site, and it still cannot reject what a pool can reach (raised 2026-08-19 against packages/apps/kubernetes/templates/cluster.yaml:21). The ask was the second call site and it is there: nodegroup.yaml:152-156 re-invokes the shared helper against the resolved per-group version. The residue is now a documentation defect rather than a missing guard, filed below as the fourth MAJOR.

Closed since my last round, verified against the code rather than the commit messages: the e2e mirror splice (the talos_spec_block helper is gone repo-wide and talos.registryMirrors is both computed and applied at talos-reconcile-job.yaml:338-343); helm.sh/resource-policy: keep on the golden (dv.yaml:17, asserted by a passing test); the tenant-string injection surface (nodegroup.yaml:135-143 now rejects an uppercase schematic, a version without the leading v, and a non-URL factory before any of the three reaches a DataVolume name); golden discoverability (two working queries at values.yaml:33-57, and the kubernetes-worker-image.cozystack.io/golden annotation they depend on is really emitted at nodegroup.yaml:248); and the missing default-sync guard (hack/check-worker-image-catalog-defaults.bats, 6 checks, auto-discovered by the hack/*.bats glob).

Claim mismatches

[PARTIAL] "Talos release outside kubernetes support window, reached either through talos.version or through image.* override" (What fails at render). The second call site exists and is fed the resolved version, but with one matrix row and a documented fail-open it can only reject a v1.13 pairing. See the fourth MAJOR for the executed render and the contradiction with _helpers.tpl. Relatedly, the guard standing in for that call site in CI is grep -c 'include "kubernetes-nodes.assertTalosSupportsKubernetes"' plus grep -q '"talosVersion" \$reqVersion' over the template text (check-worker-image-catalog-defaults.bats:96-111). It proves the call site exists and is fed the right variable, breaks on a rename, and says nothing about behaviour. Once the matrix carries a second row this becomes testable for real, and the source-text grep should give way to that case.

[PARTIAL] The release note's "Enable optional cozystack.kubernetes-worker-image package to seed one golden DataVolume per (schematicID, version) in cozy-public". True for the single shipped pair only. Per the first MAJOR there is no supported way to add a second entry, so "per (schematicID, version)" describes the template, not what an operator can reach.

Operational risks

• Deleting a golden fails the render of every pool pinned to it, and the blast radius exceeds what is broken, for the same reason as the Failed finding above. resource-policy: keep plus the operator queries in values.yaml:40-55 make an accidental delete less likely, and the printed remedy does clear the state, which is why the deletion case is here rather than in Findings. Worth deciding on purpose whether a missing golden should stop a pool's release or leave the worker DataVolume Pending with CDI reporting it.
• Every builtin worker disk in the cluster becomes a CSI clone of one source PVC in cozy-public. What LINSTOR does with N simultaneous clones of a single source, and whether clone placement is constrained to the nodes holding the source's replicas, is not established anywhere in this PR and is not something a static review can settle. It decides whether the feature scales past a handful of concurrent scale-ups, so it is worth measuring before this is recommended to anyone.

Caveats

• Not executed, needs a live cluster, out of scope here: CDI's actual clone-strategy selection for a matching StorageClass, whether the DataVolume spec is unpatchable in the way values.yaml:62-68 asserts, the golden's import against a real factory, LINSTOR clone placement and concurrency, and Server-Side Apply behaviour when the new image key first reaches a live KubernetesNodes. The "verified on dev10" note at nodegroup.yaml:228-229 is the author's observation, not something this review reproduced.
• The no-roll claim was verified rather than taken. tests/__snapshot__/render_snapshot_test.yaml.snap is untouched by the diff and its 12 snapshots pass, and a base-versus-head render diff with image omitted came back byte-identical across six value sets (defaults, talos.registryMirrors, a GPU pool, kubelet plus logSerialConsole, the podCpuLimit/podCpuRequest pair, and talos.version: v1.13.7), with the content-hash KubevirtMachineTemplate name holding at kubernetes-myk8s-md0-a6ac3e. instanceType-sized pools are not covered by the snapshot because that branch resolves through lookup; the disk-system block is identical either way, so this is a gap in the pin rather than in the conclusion.
• Checked and found sound: cross-namespace clone authorization already exists (packages/system/kubevirt-cdi/templates/cdi-cr.yaml:25-46 binds datavolumes/source create for system:serviceaccounts in cozy-public), so no new RBAC is needed; the cold-install ordering reaches the namespace creator transitively, kubernetes-worker-image to cozystack.kubevirt-cdi to cozystack.cozystack-basics, matching vm-default-images; the new golden does not appear in the tenant vm-disk source dropdown, because SourceField.tsx:7,39 filters cozy-public PVCs on the vm-default-images- prefix; the reconcile Job is content-hash named at talos-reconcile-job.yaml:478-484, so an image.*.version change produces a new Job and the in-guest installer really moves; the installer-registry asymmetry is disclosed twice, at values.yaml:122 and values.yaml:135, so an air-gapped pool is told it must also point talos.installerRepository; the new bats file is picked up by the hack/*.bats glob (Makefile:161) and the chart's test target by hack/helm-unit-tests.sh, both green (6 and 5 tests, and 102 helm-unittest cases in kubernetes-nodes); go build ./... and go vet on the touched packages are clean and zz_generated.deepcopy.go is regenerated; the new render-time regexes reject only values that could not have worked before; the cozyrds diff adds only the image property with no change to resourceNames, secrets, services or any RBAC surface; an empty or null images catalog renders nothing rather than failing; and the ghcr.io mirror deletion leaves no dangling references, with a root-cause citation for why the mirror was not the node-join fix.
• Refuted and recorded so they are not re-derived: the dashboard dropdown exposure; the cross-namespace clone RBAC; the missing spec.components being reachable by editing the Package CR by hand (helm-controller re-applies the rendered manifest and the platform chart owns the object); and a suspected coherence gap where a pool pinning image.factory.imageFactoryURL to a mirror would silently pull its installer from the public registry, which the two disclosure notes above already cover.
• The diff carries #4007's ghcr.io mirror removal, reviewed only where it touches this change: run-kubernetes.sh no longer emits a spec.talos block at all and both tenant CRs take chart defaults, which the render corners above cover.

Recommended follow-ups

• Run this on a disposable dev cluster: enable the catalog, create a builtin pool, confirm no host-assisted upload pod appears alongside the clone, and record peak pool usage across the clone-and-grow window so the sizing note in the second MAJOR can carry a real number instead of an estimate.
• vm-default-images and the other nine packages registered through cozystack.platform.package.optional share the unreachable-values gap. Giving that helper a components parameter fixes all of them at once and is a smaller change than a per-package workaround here.
• The e2e golden wait at run-kubernetes.sh:4665-4677 reports nothing when the outer timeout 12m fires before the inner kubectl wait starts, which is the case where the golden was never created because the catalog HelmRelease failed. A describe of the Package and the HelmRelease on the timeout path would name that failure instead of leaving exit 124.

Findings not anchored to changed lines

These reference code outside this PR's diff (unchanged files, or lines outside a hunk), so GitHub cannot render them inline.

[MINOR] packages/extra/computeplane/templates/cluster.yaml:207 the pool-shape allowlist was not updated and no decision was recorded

The pick list there carries a comment asking for it to be kept "in sync with the kubernetes-nodes values schema's pool-shape params", warning that a denylist "silently re-opens the hole every time kubernetes-nodes gains a new top-level value". This PR adds such a value and does not touch the file. The allowlist shape makes the outcome safe rather than broken, and the same comment argues a tenant should not pick the worker OS image, so exclusion may well be right. But image.builtin is a choice among operator-curated catalog entries, not the arbitrary URL that image.factory.imageFactoryURL is, and those are not obviously the same call. As it stands ComputePlane pools cannot use the feature, and because the NodeGroup schema has no additionalProperties: false, nodeGroups.<pool>.image.builtin on a ComputePlane CR is accepted, stored, dropped at render and never reported. Add image to the list, add it restricted to builtin, or say in the comment that it is excluded on purpose.

Verdict

NOT LGTM

Second round, at ceeec044. Everything from the previous round is closed, and I checked each one against the code rather than against the commit messages: the catalog values are reachable and I rendered the Package to prove it, the clone headroom arithmetic is in both storage and diskSize, the package header now points at KubernetesNodes and says the parent CR field is inert, talos.version's description now says what the matrix actually promises, the catalog rejects duplicate and malformed entries and refuses an edit that would need an unpatchable DataVolume patch, the enum guard fails when it cannot read the enum, the resolution logic is one shared helper, and the field is osImage. The size problem that led my last review is gone structurally: the PR is retargeted onto test/drop-ghcr-mirror, so the mirror teardown is out of the diff and the stack is declared instead of implied. Existing workers still do not roll, verified by render rather than assumed.

One thing blocks merge, and it arrived with the fix for the empty-StorageClass corner I raised last round. Resolving the empty side made the comparison possible, but the two sides are now resolved from different authorities, and the mismatch aborts a healthy pool's entire release on a cluster-wide action that pool did not take.

Closed since my last round, checked against the code

• Catalog values reachable. cozystack.platform.package.optional takes a components argument and emits the block; iaas.yaml:147-149 forwards .Values.kubernetesWorkerImage, declared at packages/core/platform/values.yaml:562. Rendering the platform with the package enabled and three overrides set put all three under spec.components.kubernetes-worker-image.values. The chart's own repository template needs a live OCIRepository, so I rendered from a copy with that template removed rather than claim a hermetic full-chart render.
• Clone headroom stated where an operator reads it, with the arithmetic: "the peak is the golden, plus diskSize + golden storage for each worker cloning at that moment", plus the failure shape, in both kubernetes-worker-image/values.yaml:78 and kubernetes-nodes/values.yaml:60, and through cozyvalues-gen into the schema and the cozyrds openAPISchema.
• The package header points at KubernetesNodes and names the parent CR field as accepted-but-inert; tests/dv_test.yaml names the right chart.
• talos.version's description now matches _helpers.tpl: the matrix rejects a pairing it lists, and an unlisted Talos minor is the operator's call. The two files no longer contradict each other.
• Duplicate and malformed catalog entries refused at dv.yaml:22-33, and dv.yaml:36-59 adds the drift guard that closes the DataVolume-immutability item I had carried open across two rounds. It names the entry, states the supported move, and says plainly that it is inert without a cluster to read.
• The enum drift guard captures first and fails on empty.
• Resolution factored into kubernetes-nodes.resolveOsImage, used by both nodegroup.yaml:96 and talos-reconcile-job.yaml:451. Behaviour-preserving: boot URL, installer image and golden name are identical to the previous head across four override shapes (builtin {}, builtin with a version, factory with a version, factory with a schematic and a mirror URL).
• image renamed to osImage. The field does not exist at the base (osImage: 0 hits in the base values.yaml, no OSImage or json:"image" in the base types), so this is an internal rename and not a CR contract change, and it propagated to values.schema.json, README.md, types.go, zz_generated.deepcopy.go, the cozyrds openAPISchema and its keysOrder, the e2e and the bats guard.
• ComputePlane records the decision: osImage is named in the exclusion comment on purpose, with the reasoning and the review reference, rather than being silently absent.
• The e2e golden wait now dumps the Package and the HelmRelease on the outer timeout, which was my third follow-up, and it captures the DataVolume list inside the until condition so a transient get cannot skip the import check.
• Existing workers do not roll: base-versus-head renders are byte-identical across five value sets (defaults, talos.registryMirrors, a GPU pool, kubelet plus logSerialConsole, talos.version: v1.13.7), and the snapshot fixtures are untouched by the diff.

Checked and found sound

• $poolCloned is a real test, not a proxy: it matches a DataVolume in the release namespace on both the pool and golden annotations and on status.phase: Succeeded, and the annotations are emitted at nodegroup.yaml:272-273. The pool-exact annotation rather than a name prefix is the right call, for the reason the comment gives about md0 and md0-gpu.
• I expected the $poolCloned gate to trade a loud render failure for a silent stall when a golden is deleted and the pool later scales up. The code answers it: the signal moves to the pending DataVolume of the worker that actually needs a clone. That is more precisely placed than a whole-release abort, and it is consistent with the blast-radius argument, so I am not filing it.
• kubernetes-nodes.defaultStorageClassName handles both the current and beta annotations, and its most-recently-created tie-break matches how Kubernetes resolves more than one default. RFC3339 timestamps are fixed-width UTC, so the string comparison is chronological as the comment claims.
• go build ./... and go vet ./api/... are clean; the generated values.schema.json, README.md and zz_generated.deepcopy.go all moved with the source.
• The three chart-lint render errors are harness artifacts, not defects: the platform chart needs a live OCIRepository, the computeplane chart requires its release to be named computeplane, and the kubernetes-nodes entry is an instanceType lookup that is empty under helm template and fails identically at the base (nodegroup.yaml:303 there, :513 here). The six missing_refs are platform-injected values, absent at the base too.
• Both template-comment-bloat flags are {{- /* */}} comments, stripped at render. The manifests this chart renders carry 88 non-# Source: comment lines at the base and 88 at this head, all of them shell comments inside the reconcile Job script.
• The values-prose-drift flag on the top-level version field is stale: the caveat about unlisted Talos minors belongs on the field that carries the override, and talos.version now has it.
• 117 helm-unittest cases and 12 snapshots green in kubernetes-nodes, 14 in kubernetes-worker-image, 6 bats checks green. shellcheck reports no errors, one warning (SC2034, an unused port at run-kubernetes.sh:4598) and the rest info or style.

Caveats

• Still out of scope for a static review and unchanged from last round: CDI's real clone-strategy selection, LINSTOR clone placement and concurrency under N simultaneous clones of one source, the golden's import against a real factory, and Server-Side Apply when the new osImage key first reaches a live KubernetesNodes. The two guards that depend on lookup are inert under helm template by construction, so the drift guard at dv.yaml:36-59 and the PVC-authority resolution are verified by their unit fixtures and by reading, not against a cluster.
• The MAJOR above was reproduced with a helm-unittest kubernetesProvider fixture, which mocks lookup responses. That is the same mechanism the chart's own suites use, so it exercises the template's real branch, but it is a model of cluster state rather than a cluster.
• Merging now depends on test/drop-ghcr-mirror landing first, since the PR is based on it.

Verdict

LGTM with non-blocking notes

Nothing blocking: the default-path render is byte-identical to the base on every corner I rendered, the new package emits nothing until it is enabled, and the guards behave as described. What is left is a set of test-coverage and message-precision gaps, plus one claim in the PR body that my own mutation pass contradicts.

Closed since my last round

My previous review was NOT LGTM at ceeec044 on three findings, all in nodegroup.yaml. One commit has landed since, 35d7a3a7, and I diffed it rather than reading its message.

• The MAJOR is closed. The cross-class guard compared the golden's frozen class against the pool's live one, so an admin moving storageclass.kubernetes.io/is-default-class failed the whole release of a pool whose disks were already correct. A pool that has already cloned now reads its class from its own bound PVC, so both sides are frozen; a pool with no disk yet is still compared against today's default, which is the class its first clone will get. The comment above the block records why gating on $poolCloned alone was rejected.
• The MINOR is closed. The message named a cluster default the render never consulted whenever the guard fired on two explicit classes. The sentence is now built only when a side was actually resolved from the default.
• The NIT is half closed, and I am re-filing the other half below. $goldenStorage now falls back to the golden's bound PVC when the DataVolume spec omits the size, which was the substance, and that fallback is asserted. The presence term itself is still unasserted.

worker_image_frozen_storageclass_test.yaml is new with the fix, 264 lines over seven cases, and it covers each of the three.

Findings

• [MINOR] packages/apps/kubernetes-nodes/templates/nodegroup.yaml:318, the written .../pool annotation value is not pinned by any test, only the reader is
• [MINOR] packages/apps/kubernetes-nodes/templates/_helpers.tpl:308, the legacy beta default-StorageClass disjunct is unreachable from any fixture
• [MINOR] packages/apps/kubernetes-nodes/templates/_helpers.tpl:267, new format guards fail-close on talos.* values the API schema still accepts and the base still rendered
• [MINOR] packages/apps/kubernetes-nodes/templates/nodegroup.yaml:264, the second remedy in the cross-class message does not do what it says
• [MINOR] packages/system/kubernetes-worker-image/templates/dv.yaml:56, the drift guard is blind exactly when the live golden carries no storageClassName
• [MINOR] packages/apps/kubernetes-nodes/values.yaml:108, version's description still names talos.version as the only matrix counterpart
• [NIT] packages/apps/kubernetes-nodes/templates/talos-reconcile-job.yaml:387, $talosVersion is now dead
• [NIT] packages/apps/kubernetes-nodes/templates/nodegroup.yaml:281, the if $goldenStorage presence term is still unasserted, carried from my last round

Claim mismatches

[PARTIAL] "mutation pass over every guard term this diff adds: each term deleted in turn, suite required to notice. Two survive, both on support-matrix guard". I ran 22 mutations across nodegroup.yaml, _helpers.tpl, talos-reconcile-job.yaml, dv.yaml and bundles/iaas.yaml and got three survivors, not two, and a fourth I found separately by hand. The second support-matrix call is one of them and it is genuinely pinned by the grep guard in hack/check-worker-image-catalog-defaults.bats:100-112, as claimed. The other two are the beta default-class disjunct and the written pool annotation above, and neither is covered by a grep guard. The fourth is the $goldenStorage presence term re-filed above, which I carried open from my previous round.

Caveats

• Verified and found sound, so it is not re-litigated: the default path does not roll workers. helm template of the pool at merge-base and at head with identical release name and values is byte-identical (31299 bytes both sides), and so are the corners talos.version=v1.13.5, talos.imageFactoryURL=<mirror>, storageClass="", instanceType=u1.medium, talos.registryMirrors, version=v1.34. The content-hashed KMT stays kubernetes-myk8s-md0-2d362e for osImage omitted, osImage: {} and osImage.factory: {}, while the pre-existing mutable fields already re-hash it (talos.version gives fdeb88, diskSize gives 6d9b26). That is also the answer to osImage being a mutable identity input: it sits in the same class as fields the chart has always accepted as mutable, and the re-image is documented in values.yaml, values.schema.json, the cozyrds openAPISchema, the README and the Go doc comment.
• Verified: the platform helper change has no blast radius on its nine other callers. Rendering packages/core/platform at merge-base and at head on values.yaml, values-isp-full.yaml and values-isp-hosted.yaml differs only by the added PackageSource; no existing Package CR changes, and none is emitted for the new package unless it is in enabledPackages.
• Verified: the pieces the clone depends on exist. datavolumes/source in cozy-public is already granted to system:serviceaccounts (packages/system/kubevirt-cdi/templates/cdi-cr.yaml:26-45), CDI v1.64.0 has no feature gate on cross-namespace clone, packages/apps/vm-disk/templates/dv.yaml:24-26 is the in-tree precedent for the same source.pvc shape out of cozy-public, and the dataVolumeTemplates annotations survive both hops (CAPK v0.1.10 pkg/kubevirt/utils.go:70-73 deep-copies the template spec and only renames, KubeVirt v1.8.4 pkg/storage/types/dv.go:125 clones the annotation map onto the created DataVolume). Flux's ServiceAccount is bound to cluster-admin (internal/fluxinstall/manifests/fluxcd.yaml:7831-7847), so the lookup calls into cozy-public and the cluster-scoped StorageClass list resolve under helm-controller.
• Mechanical sweep run on the added shell (hack/check-worker-image-catalog-defaults.bats, the changed hack/e2e-chainsaw/_lib/run-kubernetes.sh, hack/e2e-install-cozystack.bats, hack/e2e-prepare-cluster.bats). Eleven || true / 2>/dev/null hits, all refuted: the four in the new bats feed an explicit empty-check that fails the test, and the rest are diagnostic dumps or a wait loop that times out rather than passing. shellcheck on the new bats returns one SC2016 info on an intentionally single-quoted grep pattern. No RBAC was added, so the over-broad-verb sweep is vacuous here.
• Noted, not blocking: the transient storage cost of the clone has no signal above the storage layer. hack/e2e-prepare-cluster.bats:95 raises the per-node sandbox pool from 200G to 300G because the clone-and-grow ran out of space and stalled worker boot until the suites hit their node-ready timeouts. For a customer the same shape is a peak of the golden plus diskSize + golden storage per concurrent clone, multiplied by the replica count of a replicated class, and values.yaml says plainly that a backend which cannot spare it does not fail, the clone just waits and the VM never boots. That is a legal input leaving a resource pending with no surfaced condition. I am not blocking on it because the feature is opt-in, off by default, documented in diskSize and in the catalog's storage description, and reversible by flipping osImage back.
• Not executed, needs a live cluster and out of scope here: whether the CSI clone plus grow actually yields a bootable Talos root disk on a replicated class, whether the golden's PVC access mode matches what the worker disk needs, and whether helm-controller's SSA converges on a builtin pool. The render cannot answer any of these. The PR does put the clone path under the kubernetes-latest and kubernetes-previous suites and gates the pool on the golden reaching Succeeded first, which is the right coverage for it, but I have not observed that run.
• Not exercised: isp-hosted. The PackageSource renders there but the Package does not, because the iaas bundle is off on that preset. That matches kubernetes-nodes-application, which lives in the same bundle, so there is no consumer stranded on that preset.

Recommended follow-ups

• Goldens carry helm.sh/resource-policy: keep, so every Talos bump adds a new golden and leaves the previous one occupying its storage times the replica count until somebody removes it by hand. values.yaml carries the two queries that say when that is safe. Worth a scheduled check or a documented retirement step rather than relying on an operator remembering.
• ComputePlane pools cannot reach the clone path at all, by the deliberate exclusion at packages/extra/computeplane/templates/cluster.yaml:206. That is a coherent decision for a module-owned image, but it means the per-worker Factory dependency stays for those pools; worth saying so on #3950 so the platform-level default lands there too.
• Run cozystack-pr-test on a disposable dev cluster for the builtin path specifically: create the catalog, wait for the golden, create a pool on osImage.builtin, and confirm the worker disk is populated by a CSI clone with no importer or upload pod in the tenant namespace.

Findings not anchored to changed lines

These reference code outside this PR's diff (unchanged files, or lines outside a hunk), so GitHub cannot render them inline.

[MINOR] packages/apps/kubernetes-nodes/values.yaml:108 version's description still names talos.version as the only matrix counterpart

The @param {Version} version prose says the value "must satisfy the Talos<->Kubernetes support matrix against talos.version", but the render now also runs the matrix against the release osImage.builtin.version / osImage.factory.version resolves to (nodegroup.yaml:98-107).

cozyvalues-gen copies this prose verbatim into values.schema.json, the README table and the cozyrds openAPISchema the dashboard renders, so a regenerated schema does not make it current.

Not promoted: hack/check-worker-image-catalog-defaults.bats pins the shipped v1.13 row as a superset of the version enum, so no schema-valid pairing can reach that second call today. The sibling talos.version field was updated for exactly this and reads correctly.

[NIT] packages/apps/kubernetes-nodes/templates/talos-reconcile-job.yaml:387 $talosVersion is now dead

{{- $talosVersion := $.Values.talos.version }} has no reader left after line 463 switched $jobContext.talosVersion to $grpVersion. Harmless, but it reads as the value the Job uses and it is not.