Skip to content

feat(kubevirt): expose vmStateStorageClass on the KubeVirt CR - #3989

Closed
lf (lfinmauritius) wants to merge 2 commits into
cozystack:mainfrom
lfinmauritius:feat/kubevirt-vmstate-storageclass
Closed

lf (lfinmauritius) wants to merge 2 commits into
cozystack:mainfrom
lfinmauritius:feat/kubevirt-vmstate-storageclass

Conversation

@lfinmauritius

Copy link
Copy Markdown
Contributor

What this PR does

Adds vmStateStorageClass to the kubevirt chart and forwards it from the
platform's iaas bundle, so spec.configuration.vmStateStorageClass on the
KubeVirt CR becomes settable. This is the first direction #3005 lists for the
live-migration regression it reports, and today that field is unreachable from
any values file.

The problem, as #3005 states it

With persistent firmware state restored by #3154 (after #3006 had stripped it),
KubeVirt creates a persistent-state-for-<vm> PVC for every VM whose preference
sets preferredTPM.persistent / preferredEfi.persistent — which is every
windows.11, windows.2k22 and windows.2k25 preference shipped in
kubevirt-instancetypes. backend-storage.getAccessMode reads the CDI
StorageProfile of the cluster-default StorageClass, and the linstor classes
advertise ReadWriteMany for volumeMode: Block only, so the backend PVC is
created ReadWriteOnce. An RWO backend PVC pins the VM to its node: those VMs
can neither be live-migrated nor drained, while the CR asks for
evictionStrategy: LiveMigrate and workloadUpdateMethods: [LiveMigrate, Evict], so node drains and cluster upgrades can wedge on them.

Reproduced on a v1.6.1 cluster (KubeVirt v1.8.4, three nodes):

$ kubectl -n tenant-… get pvc persistent-state-for-vm-instance-<vm>
…  Bound  10Mi  RWO  local

$ kubectl -n tenant-… get vmi vm-instance-<vm> -o yaml   # .status.conditions
LiveMigratable=False  NoTSCFrequencyNotLiveMigratable
RestartRequired=True  memory updated in template spec. Memory-hotplug is only available for migratable VMs

The second condition is the user-visible cost beyond drains: because the VM is
not migratable, vmRolloutStrategy: LiveUpdate cannot hotplug CPU or memory
either. Resizing a Windows VM's instancetype is accepted, staged in the template,
and silently never applied until the VM is stopped and started.

What this changes

  • packages/system/kubevirt: vmStateStorageClass value, rendered as
    spec.configuration.vmStateStorageClass only when non-empty. Default ""
    keeps the field omitted, so behaviour is unchanged on upgrade.
  • packages/core/platform: same value forwarded onto $kubevirtValues in the
    iaas bundle when non-empty, mirroring resources.cpuAllocationRatio. Without
    this the chart value has no path: the cluster values Secret carries only the
    _cluster block.
  • make update: a sixth sed patch with its own grep guard and its own
    sanity-check line. The insert keys off the same evictionStrategy: anchor as
    permittedHostDevices and mediatedDevicesConfiguration, and a guard shared
    with either of them would skip this insert whenever that neighbour is still
    present — the exact trap tests/update_idempotency_test.sh exists to pin, so
    that suite gains the matching case.

What it deliberately does not do

It does not provision an RWX-Filesystem class, and it does not change the
default. Defaulting to replicated would not help: its ReadWriteMany is
Block-only, and the Block-mode backend volume is still a draft
(kubevirt/kubevirt#18215). So this PR delivers the lever #3005 asks for and
leaves the operator to point it at a class that advertises ReadWriteMany for
volumeMode: Filesystem — the NFS-export path the issue mentions, or any other.
Whether cozystack should ship such a class by default, and default this value to
it, looks like a separate decision and a separate PR.

Testing

  • make -C packages/system/kubevirt test — helm unittest 7/7, and
    tests/update_idempotency_test.sh PASS including the new case.
  • helm unittest packages/core/platform — 33 suites, 140 tests, all pass.
  • helm template on the chart: field absent by default, vmStateStorageClass: "nfs-csi" rendered above evictionStrategy when set.

Screenshots

N/A — no UI change.

Downstream repositories

Downstream: not ticked, one open question for a human. Walked the trigger
map file by file against the diff:

  • cozystack/website — the diff touches packages/core/platform/values.yaml,
    which the map maps to the hand-written table in
    content/en/docs/next/operations/configuration/platform-package.md. That
    table has no section for the platform's top-level virtualization keys today —
    gpu.* is absent from it as well — so the follow-up is a new section rather
    than a row, and which shape the docs owners want is not my call. Happy to open
    either the docs PR or an issue there, whichever you prefer; say which and it
    is done before this leaves draft.
  • terraform-provider-cozystack — no app values.schema.json, kind/plural,
    version enum, release prefix or CRD field is touched. No trigger.
  • ansible-cozystack — the role sets networking.* and publishing.* keys
    only; this adds a key it does not pass, and touches no installer values,
    namespace, variant or waited-on object. No trigger.
  • ccp, talm, external-apps-example, cozyhr, cozy-proxy, examples,
    cozystack-telemetry-server — nothing under hack/, no package layout,
    namespace, CRD, node prerequisite or release-prep change. No trigger.

Release note

feat(kubevirt): the KubeVirt CR's `vmStateStorageClass` is now settable, as `spec.components.platform.values.vmStateStorageClass` on the platform package and as `vmStateStorageClass` on the kubevirt component. Point it at a StorageClass that advertises `ReadWriteMany` for `volumeMode: Filesystem` to keep the persistent EFI/TPM backend PVC out of ReadWriteOnce, which otherwise makes every Windows VM (persistent TPM/EFI) non-migratable and undrainable, and blocks CPU/memory hotplug on it. Empty by default: no change on upgrade.

The chart parameterizes cpuAllocationRatio, extraFeatureGates,
permittedHostDevices and mediatedDevicesConfiguration, but nothing reaches
spec.configuration.vmStateStorageClass, so the backend-storage class for
persistent EFI/TPM state cannot be chosen at all: the one lever cozystack#3005 names for
that regression is unreachable from a values file.

Why it matters, restated from cozystack#3005: with persistent firmware state restored by
cozystack#3154, KubeVirt creates a persistent-state-for-<vm> PVC whose access mode comes
from the CDI StorageProfile of the cluster-default StorageClass. The linstor
classes advertise ReadWriteMany for volumeMode Block only, so the backend PVC is
created ReadWriteOnce, which pins the VM to its node: every VM using a windows.*
preference shipped in kubevirt-instancetypes (preferredTPM.persistent,
preferredEfi.persistent) can then neither be live-migrated nor drained, while
the CR asks for evictionStrategy: LiveMigrate and workloadUpdateMethods
[LiveMigrate, Evict].

The default stays empty, so the CR field is omitted and behaviour is unchanged
on upgrade. An operator holding a class that advertises ReadWriteMany for
volumeMode Filesystem can now point the backend at it. Nothing here provisions
such a class, and defaulting to `replicated` would not help: its RWX is
Block-only until the Block-mode backend volume of kubevirt/kubevirt#18215 lands.

The value keys off the same `evictionStrategy:` anchor as the two GPU blocks, so
`make update` gains a sixth sed patch with its own grep guard and its own
sanity-check line: a guard shared with either neighbour would skip the insert
whenever that neighbour is still present. tests/update_idempotency_test.sh gains
the matching case, dropping only this block and asserting `make update`
reinserts it without duplicating the other two.

Refs cozystack#3005, cozystack#3006, cozystack#3154

Assisted-By: Claude <[email protected]>
Signed-off-by: Loïc Fontaine <[email protected]>
The chart value added in the previous commit is unreachable without this: the
kubevirt component's values are built by the iaas bundle, and the cluster values
Secret carries only the _cluster block, so an operator has no path to
spec.configuration.vmStateStorageClass.

Same shape as resources.cpuAllocationRatio: set on $kubevirtValues only when
non-empty, so a stock render is byte-identical and the CR field stays omitted.

Refs cozystack#3005

Assisted-By: Claude <[email protected]>
Signed-off-by: Loïc Fontaine <[email protected]>
@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added area/virtualization Issues or PRs related to virtualization (kubevirt, cdi, vmi, vm-import) kind/feature Categorizes issue or PR as related to a new feature size/M This PR changes 30-99 lines, ignoring generated files labels Aug 28, 2026
Aleksei Sviridkin (lexfrei) added a commit that referenced this pull request Sep 18, 2026
#4254)

## What this PR does

Cluster-wide migration settings could only be applied by patching the
KubeVirt CR: the kubevirt chart did not render `migrations`, and the
platform bundle did not forward it. Add the optional
`kubevirt.migrations` platform value and pass it through the generated
Package to `spec.configuration.migrations`.

For example, set the following under `spec.components.platform.values`
on the `cozystack.cozystack-platform` Package:

```yaml
kubevirt:
  migrations:
    bandwidthPerMigration: 625M
    parallelMigrationsPerCluster: 2
    parallelOutboundMigrationsPerNode: 1
```

`bandwidthPerMigration` is a quantity in bytes per second. Empty or null
settings leave the field unset; existing KubeVirt defaults are
unchanged. The map is forwarded intact, including explicit `false` and
`0` values. `make update` restores the new parameterization
independently after a hand-merge.

Validation:

- All 207 Helm tests in `packages/system/kubevirt` and
`packages/core/platform` passed (42 suites).
- Before the template changes, the new limit and false/zero cases failed
at both forwarding stages (four expected regression failures).
- The GNU sed / make update-idempotency suite passed, including
restoration of a missing migrations block, repeated runs, and rejection
of a missing insertion anchor.
- Standalone KubeVirt rendering and `helm lint` passed. Neither modified
chart defines a `make generate` target. Full platform rendering requires
its live source repository lookup; the platform template behavior was
checked through its existing Helm unit-test fixtures.

This is separate from #3989, which exposes `vmStateStorageClass` in the
same chart and bundle files.


### Screenshots

Not applicable; no UI changes.

### Downstream repositories

- [ ] No downstream repository is affected by this change
- [x] [cozystack/website](https://github.com/cozystack/website) -
follow-up: [#697](cozystack/website#697)
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

The website follow-up
[#697](cozystack/website#697) documents this new
platform value in the unreleased `next` reference and should be merged
after this implementation. The merged website PR
[#678](cozystack/website#678) documents
`kubevirt.disabledFeatureGates` only. The other listed repositories do
not need a coordinated change: this adds an optional platform value
without changing app schemas, required installer values, CRDs, package
layout, source kinds, shared tooling contracts, or node prerequisites.

### Release note

```release-note
feat(kubevirt): Configure cluster-wide KubeVirt migration settings through the platform Package value kubevirt.migrations. Empty settings preserve existing defaults.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added optional KubeVirt migration configuration.
* Supports migration bandwidth, concurrency limits, auto-convergence,
and completion timeout settings.
  * Migration bandwidth can be specified in bytes per second.
  * Empty or unset migration settings preserve KubeVirt defaults.
* **Bug Fixes**
* Migration values now preserve valid `false` and `0` settings when
provided.
* **Tests**
* Added coverage for default, empty, null, configured, and
repeated-update scenarios.
* Added validation to prevent incomplete or duplicated migration
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@lexfrei

Copy link
Copy Markdown
Contributor

lf (@lfinmauritius) closing this draft. It is built on #3005, and that premise did not hold: KubeVirt migrates RWO backend storage since v1.4, and #3005 was closed on that. Without the problem there is no case for the new field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/virtualization Issues or PRs related to virtualization (kubevirt, cdi, vmi, vm-import) kind/feature Categorizes issue or PR as related to a new feature size/M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants