fix(kubernetes): default worker disk storageClass to the application storageClass - #3129
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
✅ Files skipped from review due to trivial changes (5)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughChanges the ChangesnodeGroup storageClass fallback to app-level storageClass
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the worker node storage class configuration to ensure that worker VMs receive a DRBD-backed StorageClass, which is required for live migration. By defaulting to the application-level storage class rather than the management cluster's default, the system ensures that volumes are properly configured for ReadWriteMany access. This change includes necessary updates to documentation, schema files, and unit tests to reflect the new behavior and ensure cluster stability during upgrades. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request updates the Kubernetes application package to fall back to the application-level storageClass (defaulting to replicated) when the individual nodeGroup's storageClass is left empty. This ensures that worker VM persistent disks are provisioned on a DRBD-backed StorageClass supporting ReadWriteMany (RWX) for live migration, preventing issues with linstor-csi. The changes include updates to the Helm templates, schema definitions, documentation, and comprehensive unit tests to verify the fallback logic and guard against unexpected worker-pool rolling upgrades. As there are no review comments, I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/apps/kubernetes/values.yaml`:
- Line 85: Update the StorageClass field documentation in values.yaml to remove
the implication that the fallback is always DRBD-backed. The comment for the
storageClass setting should say that worker node disks fall back to the
application-level storageClass, and that this application-level value defaults
to replicated; keep the rest of the RWX/immutability guidance intact. Use the
storageClass field description itself as the target location for the wording
change.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 73e8cb68-4d1f-4ac9-b674-ef959b32c7e4
📒 Files selected for processing (8)
api/apps/v1alpha1/kubernetes/types.gopackages/apps/kubernetes/README.mdpackages/apps/kubernetes/templates/cluster.yamlpackages/apps/kubernetes/tests/cluster_test.yamlpackages/apps/kubernetes/tests/values/common.yamlpackages/apps/kubernetes/values.schema.jsonpackages/apps/kubernetes/values.yamlpackages/system/kubernetes-rd/cozyrds/kubernetes.yaml
myasnikovdaniil
left a comment
There was a problem hiding this comment.
Summary
⚠️ MERGE CONFLICT —mergeable: CONFLICTINGagainstmain. Needs a rebase before this can land.
The fix is correct and well-tested. When a node group leaves storageClass empty, the worker DataVolume previously omitted storageClassName and fell through to the management cluster's default StorageClass, which is not guaranteed to be DRBD-backed. Since worker VMs use evictionStrategy: External, their disks must be RWX-capable, and linstor-csi only grants RWX on DRBD-backed classes. The one-liner {{- with .group.storageClass | default $.Values.storageClass }} fixes the root cause, and the both-empty opt-out path (omit storageClassName) is correctly preserved.
All documentation surfaces (types.go, values.yaml, values.schema.json, README.md, cozyrds/kubernetes.yaml) are updated in lockstep. Test coverage is thorough: six new/updated cases cover the fallback, custom app SC, group override wins, and both-empty omit path. The two hash-pin tests that document the upgrade blast radius are a nice touch.
Upgrade blast-radius note
On upgrade, CAPI will rename the KubevirtMachineTemplate for every existing tenant cluster whose md0 leaves storageClass empty — even on clusters where the management cluster default was already replicated, because the roll is driven by the body hash, not the effective StorageClass. This is safe (same class, one roll) and is clearly called out in the release note, which is the right place for it. The hash-pin tests document it in the test suite.
Blocking
- Rebase onto current
mainto resolve the merge conflict.
Non-blocking suggestions
- See inline comment on the hash-pin tests.
| documentIndex: 4 | ||
|
|
||
| ############################################### | ||
| # worker-pool roll-on-upgrade blast radius # |
There was a problem hiding this comment.
The hash-pin tests are a great mechanism for forcing deliberate attention to worker-template body changes. One quality-of-life improvement: add a brief comment above this section showing how to recompute the expected hash values when the template body changes intentionally. Otherwise future contributors will see a failing CI with a cryptic name mismatch and have to trace back to the sha256sum|trunc 6 logic in cluster.yaml.
# To recompute after an intentional template body change:
# helm template test-k8s packages/apps/kubernetes \
# -f packages/apps/kubernetes/tests/values/common.yaml \
# --set nodeGroups.md0.instanceType=u1.medium \
# | yq e 'select(.kind == "KubevirtMachineTemplate") | .metadata.name' -
# The suffix after the last '-' is the hash to update below.There was a problem hiding this comment.
Rebased onto current main and added the recompute comment you asked for, above the hash-pin section in cluster_test.yaml. I kept your helm template … | yq 'select(.kind == "KubevirtMachineTemplate") | .metadata.name' recipe with one adjustment so it reproduces the pinned hashes exactly: it renders with tests/values/common.yaml and the test-k8s release name (the same inputs the pins use) and leaves instanceType="" rather than u1.medium — setting instanceType triggers a live-cluster lookup() and changes the rendered body, so it wouldn't match …-63ff81. The comment lists both commands (default body, and --set storageClass="" for the omitted-body pin); I verified both reproduce the pins. Head is 22c96d8.
8ed9378 to
c2cf503
Compare
…storageClass Worker node VMs set evictionStrategy: External, so they live-migrate. Live migration requires the system DataVolume on a ReadWriteMany volume, and linstor-csi only grants RWX on a DRBD-backed StorageClass. When a node group leaves storageClass empty the template previously omitted the field and fell through to the management cluster default StorageClass, which may be a non-DRBD class the CSI driver rejects for RWX — breaking migration. Fall back to the application-level storageClass (the replicated/DRBD class) when the node group storageClass is empty, instead of the cluster default. When both are empty the field is still omitted, preserving the explicit cluster-default opt-out. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
c2cf503 to
22c96d8
Compare
IvanHunters
left a comment
There was a problem hiding this comment.
Verdict
LGTM
Correct, well-scoped fix: the empty-storageClass fallback for worker VM DataVolumes now targets the application-level storageClass (default replicated, a DRBD/LINSTOR class) instead of the management-cluster default, which fixes live migration on clusters whose infra default is not DRBD-backed; all four render cases are tested and pass, codegen surfaces are in lockstep, and the one-time worker-pool roll is correctly documented and guarded.
Verification performed
- Template render:
helm templatewithtests/values/common.yamlproducesstorageClassName: "replicated"on the shipped default (md0empty), template nametest-k8s-md0-63ff81matching the pin;--set storageClass=""produces nostorageClassName(nametest-k8s-md0-dcdf58, matching the omit pin). MachineDeploymentinfrastructureRefpoints at the hashed name, so the body-hash roll-on-upgrade claim is mechanically accurate. helm unittest: 140/140 pass, including the four new cases (fallback to app class, custom app class, per-node-group override wins, both-empty omit) and the two hash pins.go vet ./kubernetes/...(moduleapi/apps/v1alpha1): clean. Thetypes.gochange is a doc-comment only.- Codegen lockstep: the new
nodeGroups[].storageClassdescription is byte-identical acrosstypes.go,values.yaml,values.schema.json,README.md(padding recomputed), andpackages/system/kubernetes-rd/cozyrds/kubernetes.yaml. Schema is valid JSON, cozyrds valid YAML, working tree clean. - Semantics:
.Values.storageClassalready denotes an infra-cluster StorageClass name in existing code (templates/helmreleases/csi.yaml:34uses it asinfraStorageClass), so reusing it for the worker DataVolume (also provisioned on the infra cluster) is consistent, not a tenant-vs-infra class mix-up.
Upgrade & fresh-install impact
- Upgrade (existing tenants): one-time worker-pool roll for every tenant cluster whose
md0leavesstorageClassempty (the shipped default), driven by the template-body hash — accurate as stated, and true even for clusters already effectively onreplicated. Not data-lossy: thedisk-systemDataVolume is the ephemeral OS/kubelet/containerd disk re-imaged from the Talos factory artifact; workload data lives in separate LINSTOR-replicated tenant PVCs with independent lifecycle. The roll is disruptive (rolling worker replacement) but standard for any worker-template change. No migration script required. - Fresh install: defaulting to
replicatedcannot reference a non-existent StorageClass. The iaas bundle that provides KubeVirt (and thus tenant worker VMs) is hard-gated tobundles.system.variantin {isp-full,isp-full-generic} (packages/core/platform/templates/bundles/iaas.yaml:1-2,failotherwise), and both variants unconditionally install LINSTOR (packages/core/platform/templates/bundles/system.yaml:27and:96), which provisionsreplicated.isp-hostedlacks LINSTOR but also cannot enable iaas, so it cannot run these VMs at all. Operators who deliberately want the infra default retain the both-empty opt-out (set app-levelstorageClass: "").
What this PR does
Worker node VMs in a tenant Kubernetes cluster set
evictionStrategy: External, so they live-migrate. Live migration requires the worker system DataVolume to be ReadWriteMany, and linstor-csi only grants RWX on a DRBD-backed StorageClass. When a node group leavesstorageClassempty, the worker DataVolume template previously omittedstorageClassNameand fell through to the management cluster's default StorageClass — which is operator-configured and not guaranteed to be DRBD-backed. On any cluster whose default StorageClass is not a replicated/DRBD class, the worker disk lands on a non-RWX class and live migration breaks.This changes the empty-
storageClassfallback from the management cluster default to the application-levelstorageClass(defaultreplicated, a DRBD/LINSTOR class):{{- with .group.storageClass | default $.Values.storageClass }}. A per-node-groupstorageClassstill wins when set; when both the node group and the applicationstorageClassare empty,storageClassNameis omitted so the cluster default still applies (explicit opt-out preserved).Doc and codegen surfaces (
types.go,values.yaml,values.schema.json,README.md, and the generatedkubernetes-rdResourceDefinition) are regenerated in lockstep with the new field description. Chart unit tests cover the full contract: fallback to the application class, a custom application class, per-node-group override winning, and the both-empty omit path.Upgrade impact — one-time worker-pool roll. The
KubevirtMachineTemplatename embeds a hash of the rendered worker template body, and the MachineDeployment references that name. On the shipped default (md0leavesstorageClassempty) the body changes from "nostorageClassName" tostorageClassName: replicated, so on upgrade the template is renamed and Cluster API performs a one-time rolling replacement of the worker pool — for every existing tenant cluster, including those whose default StorageClass already wasreplicated(the roll is driven by the body hash, not by whether the effective class differs). This is the same class of roll any worker-template change triggers and is unavoidable with this approach. Two test pins document and guard the hash so future body changes are deliberate.Release note
Summary by CodeRabbit
Bug Fixes
storageClassNamefallback for empty per-node-group values: it now reliably uses the app-level replicated/DRBD StorageClass (rather than relying on the management-cluster default).Documentation
storageClass/RWX/DRBD guidance across configuration docs and schema descriptions, including the intended non-immutable behavior (optional/undefaulted).Tests