Skip to content

feat(kubernetes)!: replace ephemeral emptyDisk with persistent storage for worker nodes - #2454

Merged
Aleksei Sviridkin (lexfrei) merged 45 commits into
mainfrom
feat/kubernetes-persistent-kubelet-storage
Apr 30, 2026
Merged

feat(kubernetes)!: replace ephemeral emptyDisk with persistent storage for worker nodes#2454
Aleksei Sviridkin (lexfrei) merged 45 commits into
mainfrom
feat/kubernetes-persistent-kubelet-storage

Conversation

@Arsolitt

@Arsolitt Arsolitt (Arsolitt) commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Replace the ephemeral emptyDisk volume on tenant Kubernetes worker VMs with persistent PVC-backed storage via KubeVirt dataVolumeTemplates. Previously, kubelet certificates, kubeconfig, and containerd state were lost on VM reboot, causing the node to fall out of the tenant cluster.

Changes:

  • Rename ephemeralStorage field to diskSize in NodeGroup configuration
  • Replace emptyDisk volume with dataVolumeTemplates using CDI blank source
  • Add cdi.kubevirt.io/storage.usePopulator: "false" annotation to work around CDI volume population issues with LINSTOR CSI
  • Add optional per-nodeGroup storageClass field (defaults to cluster default)
  • Set fixed 2Gi ephemeral-storage resource for virt-launcher pod
  • Rename internal mount paths from /ephemeral to /persistent
  • Add helm unit tests (cluster_test.yaml) covering dataVolumeTemplates defaults, custom disk size, storageClass presence/absence, fixed ephemeral-storage, disk name consistency, and CDI populator annotation
  • Quote storage values in templates (diskSize, storageClassName) for Helm correctness
  • Guard bind-mount commands with mountpoint -q checks and create target directories to handle fresh images and cloud-init retries
  • Set blockSize.matchVolume on worker node disks for automatic block size detection, fixing DRBD 4Kn compatibility with QEMU

Known gotchas:

  • CDI volume population (prime PVC rebind) causes ClaimMisbound errors with LINSTOR CSI driver, regardless of StorageClass (replicated or local). The usePopulator: false annotation switches to the legacy import path which works correctly. This is a known CDI issue documented in kubevirt/containerized-data-importer#3146.
  • The replicated StorageClass with DRBD suspend-io quorum policy can cause transient VM IO errors during LINSTOR satellite restarts (quorum loss). The local StorageClass is recommended for worker node disks. When storageClass is not specified in a nodeGroup, the cluster default StorageClass is used.
  • Existing clusters will trigger a rolling replacement of all worker nodes on upgrade due to changed KubevirtMachineTemplate hash.
  • DRBD volumes use 4096-byte logical sectors (block-size 4096). Without explicit block size configuration, QEMU defaults to 512-byte sectors, causing immediate IO errors and VM pause. The blockSize.matchVolume setting in the disk spec ensures KubeVirt auto-detects the correct sector size from the volume.

Tested on a live cluster:

  • Verified DataVolume provisioning with local StorageClass
  • Verified nodes join the tenant cluster and reach Ready state
  • Verified node survives VM reboot (pod deletion) and returns to Ready with the same Node object, reusing kubelet certificates from persistent disk

Screenshots

N/A — no UI changes.

Release note

feat(kubernetes)!: replace ephemeral emptyDisk with persistent storage for worker nodes

Worker node VMs now use persistent PVC-backed disks instead of ephemeral emptyDisk volumes. Kubelet certificates, kubeconfig, and containerd state survive VM reboots, preventing nodes from falling out of the tenant cluster.

BREAKING CHANGE: The `ephemeralStorage` field in nodeGroups has been renamed to `diskSize`. A new optional `storageClass` field allows per-nodeGroup storage class selection. Users must update their values overrides accordingly.

Worker node disks now use `blockSize.matchVolume` to automatically detect the underlying volume's sector size, ensuring compatibility with DRBD-backed storage classes that use 4096-byte logical sectors.

Summary by CodeRabbit

  • New Features

    • Optional storageClass parameter to select storage class for worker node persistent disks (defaults to cluster default).
    • Makefile test target added to run Helm unit tests for the chart.
  • Configuration Changes

    • Worker node storage parameter renamed from ephemeralStorage to diskSize (default: 20Gi).
    • Kubelet/containerd state moved to persistent disk paths and node ephemeral-storage accounting fixed to 2Gi; disk provisioning aligns block sizing with backing volumes.
  • Documentation

    • Updated docs and chart/schema to reflect the persistent disk model and new fields.

@coderabbitai

coderabbitai Bot commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Replaces ephemeral worker disks with CDI-backed persistent DataVolumes and renames NodeGroup ephemeralStoragediskSize, adds optional storageClass, updates Helm templates/values/schema/tests, and adjusts kubeadm/mount paths and pod resource requests to use persistent paths and a fixed ephemeral-storage: "2Gi".

Changes

Cohort / File(s) Summary
CRD Type Definitions
api/apps/v1alpha1/kubernetes/types.go, api/apps/v1alpha1/kubernetes/zz_generated.deepcopy.go
Replaced NodeGroup.ephemeralStorage (JSON ephemeralStorage) with NodeGroup.diskSize (JSON diskSize) and added NodeGroup.storageClass (JSON storageClass,omitempty); updated deepcopy to copy DiskSize.
Helm Chart Build & Docs
packages/apps/kubernetes/Makefile, packages/apps/kubernetes/README.md
Added test make target (helm unittest .); README updated to document diskSize, storageClass, and blockSize.matchVolume behavior.
Helm Storage Template
packages/apps/kubernetes/templates/cluster.yaml
Converted VM second disk from emptyDisk to CDI dataVolumeTemplate (disk-kubelet), enabled blockSize.matchVolume, switched kubeadm/mounts from /ephemeral/persistent, and set pod ephemeral-storage request to "2Gi".
Helm Template Tests
packages/apps/kubernetes/tests/cluster_test.yaml
Added Helm unit tests verifying disk-kubelet DataVolume template, default/override diskSize (20Gi default), storageClass omit vs set behavior, consistent naming across references, CDI annotation cdi.kubevirt.io/storage.usePopulator: "false", and blockSize.matchVolume enabled.
Helm Configuration Schemas & Defaults
packages/apps/kubernetes/values.schema.json, packages/apps/kubernetes/values.yaml
Renamed nodeGroups.*.ephemeralStoragenodeGroups.*.diskSize, added nodeGroups.*.storageClass, updated defaults (md0 → diskSize: 20Gi) and descriptions/validation.
Embedded Chart Schema
packages/system/kubernetes-rd/cozyrds/kubernetes.yaml
Updated embedded OpenAPI/chart-values schema and keysOrder to replace ephemeralStorage with diskSize and add storageClass under nodeGroups.

Sequence Diagram(s)

sequenceDiagram
  participant User as Chart values
  participant Helm as Helm renderer
  participant CDI as CDI (DataVolume controller)
  participant VM as KubeVirt VM / kubelet

  User->>Helm: provide nodeGroups.diskSize & storageClass
  Helm->>CDI: render dataVolumeTemplate "disk-kubelet" (size/class, annotations)
  Helm->>VM: render VM spec referencing DataVolume "disk-kubelet" and blockSize.matchVolume
  CDI->>VM: provision DataVolume backing disk (DataVolume created/populated)
  VM->>VM: kubeadm scripts mount /persistent and configure kubelet/containerd
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 I swapped my burrow's sand for a steadfast disk,

DataVolumes hum while my paws do a brisk frisk,
diskSize declared, storageClass may nod,
Persistent crumbs tucked under a solid sod,
A rabbit hops — stable storage, what a whisk!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The pull request title accurately and concisely summarizes the primary change: replacing ephemeral emptyDisk volumes with persistent storage for worker nodes in Kubernetes, which is the core focus of the entire changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/kubernetes-persistent-kubelet-storage

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Arsolitt
Arsolitt (Arsolitt) marked this pull request as ready for review April 23, 2026 07:59
@dosubot dosubot Bot added size/S This PR changes 10-29 lines, ignoring generated files kind/feature Categorizes issue or PR as related to a new feature labels Apr 23, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request improves the stability of tenant Kubernetes worker nodes by transitioning from ephemeral storage to persistent PVC-backed disks. This change ensures that critical node state, such as kubelet certificates and containerd data, is preserved across VM reboots, preventing nodes from unexpectedly leaving the cluster. The update includes necessary configuration schema changes and adds robust unit testing to verify the new storage implementation.

Highlights

  • Persistent Storage Migration: Replaced ephemeral emptyDisk volumes with persistent PVC-backed storage using KubeVirt dataVolumeTemplates to ensure kubelet certificates and containerd state survive VM reboots.
  • Configuration Changes: Renamed ephemeralStorage to diskSize in NodeGroup configuration and added an optional storageClass field for per-nodeGroup storage selection.
  • Workaround for CDI/LINSTOR: Added cdi.kubevirt.io/storage.usePopulator: "false" annotation to resolve volume population issues with the LINSTOR CSI driver.
  • Testing: Introduced comprehensive helm unit tests in cluster_test.yaml to validate dataVolumeTemplates, disk size, storageClass, and annotation settings.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files
  • Ignored by pattern: **/zz_generated.*.go (1)
    • api/apps/v1alpha1/kubernetes/zz_generated.deepcopy.go
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@Arsolitt Arsolitt (Arsolitt) self-assigned this Apr 23, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request renames ephemeralStorage to diskSize and introduces a storageClass field for worker node persistent disks in the Kubernetes package. The implementation transitions from using emptyDisk to dataVolumeTemplates to ensure data persistence for kubelet and containerd. The changes include updates to API types, Helm templates, documentation, and the addition of a new test suite using helm unittest. Feedback was provided to ensure Helm template correctness by using the quote function for storage values and storage class names, as per the project style guide.

storage:
resources:
requests:
storage: {{ .group.diskSize | default "20Gi" }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

It is recommended to use the quote function for storage values in Helm templates to ensure they are always treated as strings, especially if the value could be interpreted as a number.

                storage: {{ .group.diskSize | default "20Gi" | quote }}
References
  1. Helm template correctness: missing quote (link)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed — storage uses | quote since commit 21a8c95e:

storage: {{ .group.diskSize | default "20Gi" | quote }}

requests:
storage: {{ .group.diskSize | default "20Gi" }}
{{- with .group.storageClass }}
storageClassName: {{ . }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The storageClassName value should be quoted to ensure it is correctly parsed as a string in the generated manifest.

            storageClassName: {{ . | quote }}
References
  1. Helm template correctness: missing quote (link)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed — storageClassName uses | quote since commit 21a8c95e:

storageClassName: {{ . | quote }}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/apps/kubernetes/tests/cluster_test.yaml (1)

20-418: Optional: factor out duplicated test fixtures.

Each of the seven test cases repeats the same ~30 lines of release, _namespace, controlPlane, and nodeGroup boilerplate, with only 1–2 fields actually varying per case. Consider extracting the common values into a shared YAML file and referencing via values: (or a YAML anchor with <<: *base) so each test only expresses its delta. This makes the intent of each case obvious and keeps future schema changes from requiring seven identical edits.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/apps/kubernetes/tests/cluster_test.yaml` around lines 20 - 418, The
test file repeats the same release/_namespace/controlPlane/nodeGroups (md0)
fixture across multiple "it" cases; extract the common block into a shared
values object and reference it from each test (e.g., create a base YAML mapping
called baseValues containing release, _namespace, version, controlPlane, and
nodeGroups.md0) or use a YAML anchor (<<: *base) and then override only the
differing keys (diskSize, storageClass, etc.) in each "it" case so tests like
"renders dataVolumeTemplate with default 20Gi storage", "renders
dataVolumeTemplate with custom 50Gi storage", and the other cases only contain
the deltas.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/system/kubernetes-rd/cozyrds/kubernetes.yaml`:
- Line 29: The keysOrder array is missing the per-nodeGroup storage key
spec.nodeGroups.md0.storageClass, so add
["spec","nodeGroups","md0","storageClass"] into the keysOrder list immediately
after ["spec","nodeGroups","md0","diskSize"] so the new per-nodeGroup storage
option is ordered with the other worker storage settings; update the keysOrder
entry in kubernetes.yaml (look for the keysOrder array and the nearby
["spec","nodeGroups","md0","diskSize"] entry) to include this new key.

---

Nitpick comments:
In `@packages/apps/kubernetes/tests/cluster_test.yaml`:
- Around line 20-418: The test file repeats the same
release/_namespace/controlPlane/nodeGroups (md0) fixture across multiple "it"
cases; extract the common block into a shared values object and reference it
from each test (e.g., create a base YAML mapping called baseValues containing
release, _namespace, version, controlPlane, and nodeGroups.md0) or use a YAML
anchor (<<: *base) and then override only the differing keys (diskSize,
storageClass, etc.) in each "it" case so tests like "renders dataVolumeTemplate
with default 20Gi storage", "renders dataVolumeTemplate with custom 50Gi
storage", and the other cases only contain the deltas.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23ca0f22-241b-4852-a17b-ea7d6a7a4df0

📥 Commits

Reviewing files that changed from the base of the PR and between 7fff77f and 6086be1.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (9)
  • api/apps/v1alpha1/kubernetes/types.go
  • api/apps/v1alpha1/kubernetes/zz_generated.deepcopy.go
  • packages/apps/kubernetes/Makefile
  • packages/apps/kubernetes/README.md
  • packages/apps/kubernetes/templates/cluster.yaml
  • packages/apps/kubernetes/tests/cluster_test.yaml
  • packages/apps/kubernetes/values.schema.json
  • packages/apps/kubernetes/values.yaml
  • packages/system/kubernetes-rd/cozyrds/kubernetes.yaml

Comment thread packages/system/kubernetes-rd/cozyrds/kubernetes.yaml Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/apps/kubernetes/templates/cluster.yaml`:
- Around line 247-249: The cloud-init mounts list includes bind mounts for
/var/lib/kubelet and /var/lib/containerd but the target directories may not
exist, causing mount failures before preKubeadmCommands run; update the
cloud-init config to create both source (e.g., /persistent/kubelet,
/persistent/containerd) and target directories (e.g., /var/lib/kubelet,
/var/lib/containerd) prior to performing mounts (use mkdir -p for targets), and
replace direct mount entries with idempotent mount logic that checks mountpoint
(or use a small shell snippet invoked before mounts) to avoid duplicate mounts
and handle timing variations; reference the mounts section entries and the
preKubeadmCommands block when adding these mkdir -p and mountpoint checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ba24933a-bd8b-442e-8822-4e2b3153eb0d

📥 Commits

Reviewing files that changed from the base of the PR and between 6086be1 and 9e1171c.

📒 Files selected for processing (2)
  • packages/apps/kubernetes/templates/cluster.yaml
  • packages/apps/kubernetes/tests/cluster_test.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/apps/kubernetes/tests/cluster_test.yaml

Comment thread packages/apps/kubernetes/templates/cluster.yaml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
packages/system/kubernetes-rd/cozyrds/kubernetes.yaml (1)

29-29: ⚠️ Potential issue | 🟡 Minor

Add the per-nodeGroup storageClass to keysOrder.

Line 29 orders diskSize but skips the new spec.nodeGroups.md0.storageClass, so the dashboard will not place this worker disk option with the rest of the node group storage settings.

Suggested insertion
- ["spec", "nodeGroups", "md0", "diskSize"], ["spec", "nodeGroups", "md0", "roles"]
+ ["spec", "nodeGroups", "md0", "diskSize"], ["spec", "nodeGroups", "md0", "storageClass"], ["spec", "nodeGroups", "md0", "roles"]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kubernetes-rd/cozyrds/kubernetes.yaml` at line 29, The
keysOrder array is missing the per-nodeGroup storageClass entry so the dashboard
won't group storage settings with the node group; update keysOrder to include
["spec","nodeGroups","md0","storageClass"] adjacent to the existing
["spec","nodeGroups","md0","diskSize"] entry (i.e., insert the
["spec","nodeGroups","md0","storageClass"] key immediately before or after the
["spec","nodeGroups","md0","diskSize"] element in the keysOrder array) so
spec.nodeGroups.md0.storageClass is ordered with the node group's disk settings.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@packages/system/kubernetes-rd/cozyrds/kubernetes.yaml`:
- Line 29: The keysOrder array is missing the per-nodeGroup storageClass entry
so the dashboard won't group storage settings with the node group; update
keysOrder to include ["spec","nodeGroups","md0","storageClass"] adjacent to the
existing ["spec","nodeGroups","md0","diskSize"] entry (i.e., insert the
["spec","nodeGroups","md0","storageClass"] key immediately before or after the
["spec","nodeGroups","md0","diskSize"] element in the keysOrder array) so
spec.nodeGroups.md0.storageClass is ordered with the node group's disk settings.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6cb3e079-636a-44da-a897-c5b0d24ca181

📥 Commits

Reviewing files that changed from the base of the PR and between da6679b and 0c20a36.

📒 Files selected for processing (1)
  • packages/system/kubernetes-rd/cozyrds/kubernetes.yaml

…config

Rename the ephemeralStorage field to diskSize in the NodeGroup typedef
and default values to reflect the upcoming switch from ephemeral
emptyDisk volumes to persistent dataVolumeTemplates for worker nodes.

BREAKING CHANGE: The nodeGroups.*.ephemeralStorage field has been renamed
to nodeGroups.*.diskSize. Users must update their values overrides.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
…tes for worker nodes

Replace the ephemeral emptyDisk volume with a CDI dataVolumeTemplate
backed by the tenant's storageClass. This ensures kubelet certificates,
kubeconfig, and containerd state survive VM reboots, preventing worker
nodes from falling out of the cluster.

Changes in the KubevirtMachineTemplate:
- Add dataVolumeTemplates with a blank source disk sized by diskSize
- Switch the volume reference from emptyDisk to dataVolume
- Rename disk and volume from "ephemeral" to "disk-kubelet"

Changes in the KubeadmConfigTemplate:
- Rename filesystem label from "ephemeral" to "persistent"
- Update all mount paths from /ephemeral to /persistent

BREAKING CHANGE: Worker node VMs now use persistent PVC-backed disks
instead of ephemeral emptyDisk volumes. Existing clusters will trigger
a rolling replacement of worker nodes on upgrade.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
With PVC-backed persistent disks, the virt-launcher pod no longer needs
ephemeral-storage sized to the full disk. Set a fixed 2Gi allocation
for QEMU runtime needs and proper scheduler behavior.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add cdi.kubevirt.io/storage.usePopulator: "false" annotation to
the dataVolumeTemplate to prevent CDI volume population flow.

The populator mechanism creates an intermediate "prime" PVC and
rebinds the PV, which causes ClaimMisbound errors with LINSTOR CSI.
The legacy import path writes directly to the target PVC.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Allow each node group to specify its own storageClass for persistent
worker node disks. When empty, the cluster default StorageClass is used.
This enables using local storage for worker disks while keeping
replicated storage for tenant workloads.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
…lues

Signed-off-by: Arsolitt <arsolitt@gmail.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
The helm unit tests in tests/cluster_test.yaml were silently skipped by CI
because the Makefile had no test target. CI runs hack/helm-unit-tests.sh
which only invokes make test when the target exists.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
…r output

Signed-off-by: Arsolitt <arsolitt@gmail.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
…Kn compatibility

Signed-off-by: Arsolitt <arsolitt@gmail.com>
…ode disks

Signed-off-by: Arsolitt <arsolitt@gmail.com>
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 24, 2026
DataVolumes are tied to VM lifecycle — PVC does not survive VM
recreation. The real reason for overwrite: false is the same-VM
reboot path: XFS on /dev/vdb already exists and cloud-init must
skip reformatting.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Add explicit scope: persistence covers same-VM reboots only.
VM replacement by CAPI still provisions a fresh PVC.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
Specify that empty storageClass uses the management cluster default
StorageClass (annotated with is-default-class: true), not the
top-level chart storageClass value.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Arsolitt <arsolitt@gmail.com>
@dosubot dosubot Bot added size/M This PR changes 30-99 lines, ignoring generated files and removed size/S This PR changes 10-29 lines, ignoring generated files labels Apr 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough follow-ups — all three items addressed precisely:

  • overwrite: false comment now reflects the actual same-VM reboot reasoning (e2b10d42).
  • Breaking Changes note clarifies persistence scope: same-VM reboots vs CAPI VM replacement (3afcaa5d).
  • storageClass description now specifies the management cluster default StorageClass with the is-default-class: true annotation (cd4590b4).

LGTM.

Incorporate images.waitForKubeconfig support from main while
keeping the persistent-storage diskSize and storageClass fields
introduced by this branch.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
IvanHunters
IvanHunters previously approved these changes Apr 27, 2026
@Arsolitt Arsolitt (Arsolitt) added the do-not-merge/hold Indicates that a PR should not merge because someone has issued /hold label Apr 28, 2026
Integrate the HAMi GPU virtualization addon from main while
keeping the persistent-storage diskSize and storageClass fields
introduced by this branch.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
@github-actions github-actions Bot added area/kubernetes Issues or PRs related to the tenant Kubernetes app kind/breaking-change Indicates the change introduces a breaking API or behaviour change labels Apr 29, 2026
@Arsolitt Arsolitt (Arsolitt) removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued /hold label Apr 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-verified the PR substance after the merge from main.

Problem and scope: Real and well-scoped — kubelet client/serving certs in /var/lib/kubelet/pki and containerd state are wiped on emptyDisk-backed VM reboot, dropping the worker out of the tenant cluster. Reproducible, well-described.

Approach vs alternatives: DataVolumeTemplate with blank source is the right pick for the CAPI+KubeVirt model. Stateless re-bootstrap on every reboot (Talos-style) would require long-lived bootstrap tokens, CSR auto-approval, and a redesigned KubeadmConfigTemplate — out of scope. PVC referencing instead of dataVolumeTemplate breaks per-VM lifecycle management. kubelet cert rotation only rotates existing certs and cannot recover from full loss. The storage: (vs pvc:) spec, blockSize.matchVolume for DRBD 4Kn, usePopulator: false workaround for the LINSTOR CSI + CDI populator interaction, fixed 2Gi virt-launcher ephemeral-storage, and the mountpoint -q / mkdir -p idempotency guards in cloud-init all line up correctly with the model.

Acknowledged tradeoffs: Rolling replacement of all workers on upgrade (KubevirtMachineTemplate hash change), state lost on CAPI VM replacement, manual PVC deletion is break-glass — all named in the README.

Two non-blocking observations worth raising for follow-up (do not need to address before merge):

  1. Default StorageClass for worker disks may be a regression in resource efficiency. The README itself notes "The local StorageClass is recommended for worker node disks" because kubelet/containerd state is local-only by nature, replication has no failover use case here, and CAPI provisions a fresh PVC on VM replacement anyway. With storageClass: "" the chart falls back to the cluster-default SC, which on typical cozystack clusters is replicated — so the default deployment pays DRBD replication cost (2-3x storage + network) for no benefit. Worth either documenting storageClass: local as the recommended override in values.yaml next to the field, or splitting "tenant data SC" from "worker disk SC" at the chart level. Idiomatic-Kubernetes-wise, deferring to the operator's default-class annotation is defensible — flagging for visibility, not as a fix request.

  2. Silent CRD field drop on direct kubectl path. The Helm migration guard fails loudly on ephemeralStorage, but kubectl apply -f kubernetes-cr.yaml with the legacy field name silently drops it and reverts to the 20Gi default — README acknowledges this. A CEL validation rule on the CRD (e.g., rejecting unknown legacy field names with a helpful message) would close the gap. Not blocking the breaking-change bump since the rename is signposted in the release-note and README.

LGTM.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit dff5f8d into main Apr 30, 2026
9 checks passed
@lexfrei
Aleksei Sviridkin (lexfrei) deleted the feat/kubernetes-persistent-kubelet-storage branch April 30, 2026 09:41
Arsolitt (Arsolitt) added a commit that referenced this pull request Apr 30, 2026
Merge origin/main into feat/kubernetes-kubelet-reserved-resources.

PR #2454 (persistent storage for worker nodes) landed in main and
touched the same nodeGroups schema. Conflicts resolved by keeping
both feature sets (kubelet reservations + persistent storage):

- api/apps/v1alpha1/kubernetes/types.go: merged kubebuilder default
  marker with all keys from both features
- packages/apps/kubernetes/README.md: merged Parameters table with
  diskSize/storageClass + kubelet* rows
- packages/system/kubernetes-rd/cozyrds/kubernetes.yaml: merged
  openAPISchema and keysOrder, dropped stale ephemeralStorage path

Signed-off-by: Arsolitt <arsolitt@gmail.com>
Arsolitt (Arsolitt) added a commit that referenced this pull request Apr 30, 2026
PR #2454 renamed the nodeGroups field ephemeralStorage to diskSize and
added a migration guard that fails template rendering when the old name
is used. Update kubelet reservation tests to use the new field name.

Signed-off-by: Arsolitt <arsolitt@gmail.com>
IvanHunters added a commit that referenced this pull request May 20, 2026
Add pre-upgrade migration 40 that walks all kuberneteses.apps.cozystack.io
Application CRs and renames nodeGroups[*].ephemeralStorage to diskSize,
preserving the user's value.

Without this migration, clusters upgraded after PR #2454 either fail to
reconcile (hard fail blocks all Flux operations) or silently lose the user's
disk size setting (default 20Gi replaces whatever was configured).

The migration is idempotent and best-effort: a failed patch is logged and
leaves the version stamp at 40 for retry on next upgrade.

Update the chart-side guard error message to direct operators to the
migration job logs when the field appears post-upgrade (regression detector).
Bump migrations.targetVersion 40 -> 41.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
myasnikovdaniil pushed a commit that referenced this pull request May 21, 2026
Add pre-upgrade migration 41 that walks all kuberneteses.apps.cozystack.io
Application CRs and renames nodeGroups[*].ephemeralStorage to diskSize,
preserving the user's value.

Without this migration, clusters upgraded after PR #2454 either fail to
reconcile (hard fail blocks all Flux operations) or silently lose the user's
disk size setting (default 20Gi replaces whatever was configured).

The migration is idempotent and best-effort: a failed patch is logged and
leaves the version stamp at 41 for retry on next upgrade.

Update the chart-side guard error message to direct operators to the
migration job logs when the field appears post-upgrade (regression detector).
Bump migrations.targetVersion 41 -> 42.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
myasnikovdaniil added a commit that referenced this pull request May 21, 2026
…ook (#2688)

## Problem

PR #2454 renamed `nodeGroups[*].ephemeralStorage` →
`nodeGroups[*].diskSize` with
a hard `{{ fail }}` guard. Any cluster whose HelmRelease still carries
the legacy
field cannot be reconciled by Flux at all — unrelated control-plane
changes and
MachineHealthCheck remediations are also blocked.

## Solution

Add platform migration 41 that runs as a pre-upgrade hook before any
chart
resources are applied. The migration walks every
`kuberneteses.apps.cozystack.io`
Application CR cluster-wide and renames `nodeGroups[*].ephemeralStorage`
to
`nodeGroups[*].diskSize`, preserving the user's value.

- Idempotent: a second run is a no-op (field is already absent)
- Best-effort: a failed patch is logged; the version stamp stays at 41
so the
  migration retries on the next platform upgrade
- Bumps `migrations.targetVersion` 41 → 42

The chart-side guard remains in place with an updated error message that
directs
operators to the migration job logs if the field somehow reappears
post-upgrade.

## Testing

Verified on dev cluster: migration correctly renames `ephemeralStorage`
values
in existing Application CRs. All 123 helm unit tests pass.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **Documentation**
* Platform migration 41 now automatically handles the `ephemeralStorage`
to `diskSize` field rename for Kubernetes node groups
* Migration is transparent during upgrade with no manual configuration
changes required
  * Error messages updated with clearer troubleshooting guidance

* **Tests**
  * Updated test assertions for migration error messages

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2688?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
myasnikovdaniil added a commit that referenced this pull request May 22, 2026
…kSize via pre-upgrade hook (#2712)

Add pre-upgrade migration 41 that walks all
kuberneteses.apps.cozystack.io Application CRs and renames
nodeGroups[*].ephemeralStorage to diskSize, preserving the user's value.

Without this migration, clusters upgraded after PR #2454 either fail to
reconcile (hard fail blocks all Flux operations) or silently lose the
user's disk size setting (default 20Gi replaces whatever was
configured).

The migration is idempotent and best-effort: a failed patch is logged
and leaves the version stamp at 41 for retry on next upgrade.

Update the chart-side guard error message to direct operators to the
migration job logs when the field appears post-upgrade (regression
detector). Bump migrations.targetVersion 41 -> 42.

Assisted-By: Claude <noreply@anthropic.com>

(cherry picked from commit ed1bb53)

<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->

## What this PR does


### Screenshots

<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->

### Release note

<!--  Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->

```release-note

```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/kubernetes Issues or PRs related to the tenant Kubernetes app kind/breaking-change Indicates the change introduces a breaking API or behaviour change kind/feature Categorizes issue or PR as related to a new feature lgtm This PR has been approved by a maintainer size/M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants