Skip to content

feat/impruvement-kubernetes-tests - #1485

Merged
Andrei Kvapil (kvaps) merged 6 commits into
mainfrom
feat/impruvement-kubernetes-tests
Oct 6, 2025
Merged

feat/impruvement-kubernetes-tests#1485
Andrei Kvapil (kvaps) merged 6 commits into
mainfrom
feat/impruvement-kubernetes-tests

Conversation

@IvanHunters

@IvanHunters IvanHunters commented Oct 3, 2025

Copy link
Copy Markdown
Collaborator

What this PR does

Release note

Improved tests for verifying installed kubernetes client clusters

Summary by CodeRabbit

  • New Features

    • Added node readiness checks (expected node count, detailed node display) and kubelet version validation with compatibility handling.
  • Improvements

    • Increased API port-forward timeout and extended rollout/machine deployment waits for more reliable rollouts.
    • Added per-component readiness waits for core cluster services.
  • Chores

    • Bumped default Kubernetes version to v1.33.

@coderabbitai

coderabbitai Bot commented Oct 3, 2025

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

IvanHunters has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 10 minutes and 40 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between f2cd585 and 012906c.

📒 Files selected for processing (1)
  • hack/e2e-apps/run-kubernetes.sh (1 hunks)

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Script extends e2e Kubernetes validation: increases kube-apiserver port-forward timeout, adds node readiness and kubeletVersion checks, adds per-component HelmRelease waits, and lengthens machine deployment replicas wait. Makefile bumps KUBERNETES_VERSION v1.32 → v1.33. No public APIs changed. (≤50 words)

Changes

Cohort / File(s) Summary
E2E validation and waits
hack/e2e-apps/run-kubernetes.sh
Increased kube-apiserver port-forward timeout (40s → 200s); added wait for 2 ready nodes (2m) and kubectl get nodes -o wide; verify kubeletVersion across nodes with special handling for v1.32 vs v1.33 and fail on mismatch; added per-component HelmRelease readiness waits for cilium, coredns, csi, ingress-nginx, vsnap-crd (each 1m); replaced shorter machineDeployment wait with wait for all replicas ready (10m) using v1beta2 readyReplicas.
Kubernetes version bump
packages/apps/kubernetes/Makefile
Updated KUBERNETES_VERSION from v1.32 to v1.33.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Runner as Dev/CI
  participant Script as run-kubernetes.sh
  participant API as kube-apiserver (port-forward)
  participant Cluster as Kubernetes
  participant HR as HelmRelease controller

  Runner->>Script: start e2e script
  Script->>API: port-forward (timeout 200s)
  API-->>Script: connection established

  Script->>Cluster: wait for 2 Nodes ready (2m)
  Cluster-->>Script: nodes list (wide)
  Script->>Cluster: verify kubeletVersion across nodes
  alt kubelet versions mismatch
    Cluster-->>Script: report mismatch
    Script-->>Runner: exit with error
  else kubelet versions match
    Script->>HR: wait cilium Ready (1m)
    Script->>HR: wait coredns Ready (1m)
    Script->>HR: wait csi Ready (1m)
    Script->>HR: wait ingress-nginx Ready (1m)
    Script->>HR: wait vsnap-crd Ready (1m)
    Script->>Cluster: wait machinedeployment readyReplicas (10m)
    Cluster-->>Script: all ready / timeout
    Script-->>Runner: proceed or fail
  end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

  • kvaps
  • lllamnyp
  • klinch0

Poem

I hop through logs with whiskers bright,
Two nodes stand ready in morning light.
Kubelets checked, versions align,
Helm charts settle, all looks fine.
A bumped version, hopping on — joy and a carrot sign. 🥕🐇

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Title Check ⚠️ Warning The title reads like a branch name with a misspelling and does not form a clear, concise sentence summarizing the primary change, so it fails to communicate the intent effectively. Please rename the pull request title to a clear, natural-language summary such as “Improve Kubernetes client cluster verification tests,” correcting the spelling and removing slashes and prefixes.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello IvanHunters, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the stability and maintainability of Kubernetes deployments and testing. It incorporates an upgrade to the latest Kubernetes version, enhances the reliability of end-to-end tests by adding more thorough validation steps, and refines the installer build process for better platform compatibility.

Highlights

  • Kubernetes Version Upgrade: The default Kubernetes version used in the project has been updated from v1.32 to v1.33.
  • Enhanced E2E Test Robustness: End-to-end tests for Kubernetes clusters now include more comprehensive checks, such as waiting for nodes to be ready, verifying kubelet versions, and ensuring critical HelmReleases (Cilium, CoreDNS, CSI, Ingress-Nginx, Vsnap-CRD) are in a ready state. The port-forwarding timeout was also increased for stability.
  • Installer Build Platform Specification: The Docker buildx command for the image-cozystack installer now explicitly specifies linux/amd64 as the target platform, ensuring consistent builds.
  • Default ETCD for Tenant Apps: The etcd parameter in the tenant application values is now set to true by default.
  • Installer Image Configuration Streamlining: A file containing a hardcoded installer image definition (packages/core/installer/values.yaml) has been removed, likely simplifying image management and configuration.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves the Kubernetes e2e tests by adding more verification steps, such as checking node readiness and kubelet versions. The changes are a good step towards more robust testing. I've provided a few suggestions to fix a bug in the version check logic and to improve the script's maintainability and robustness. As a minor note, there's a typo in the pull request title: 'impruvement' should be 'improvement'.

Comment thread hack/e2e-apps/run-kubernetes.sh Outdated
Comment on lines +107 to +109
if [[ ! "$v" =~ ^"${k8s_version}"[^$]+ ]]; then
node_ok=false
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The regex used for verifying the kubelet version is incorrect. The pattern ^\"${k8s_version}\"[^$]+ requires at least one character to follow the version string, which means an exact version match will fail this check. For example, if ${k8s_version} is v1.33.0 and the kubelet version is also v1.33.0, the check will incorrectly fail.

I've suggested a fix that corrects the regex and adds a break to exit the loop on the first failure, making it more efficient.

Suggested change
if [[ ! "$v" =~ ^"${k8s_version}"[^$]+ ]]; then
node_ok=false
fi
if [[ ! "$v" =~ ^"${k8s_version}" ]]; then
node_ok=false
break
fi


# Wait for the nodes to be ready (timeout after 2 minutes)
timeout 2m bash -c '
until [ "$(kubectl --kubeconfig tenantkubeconfig get nodes -o jsonpath="{.items[*].metadata.name}" | wc -w)" -eq 2 ]; do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using wc -w to count nodes can be brittle. For example, it would miscount if there were extra spaces between node names. A more robust approach is to squeeze spaces, convert them to newlines with tr -s, and then count the lines with wc -l.

Suggested change
until [ "$(kubectl --kubeconfig tenantkubeconfig get nodes -o jsonpath="{.items[*].metadata.name}" | wc -w)" -eq 2 ]; do
until [ "$(kubectl --kubeconfig tenantkubeconfig get nodes -o jsonpath="{.items[*].metadata.name}" | tr -s ' ' '\n' | wc -l)" -eq 2 ]; do

Comment thread hack/e2e-apps/run-kubernetes.sh Outdated
Comment on lines +121 to +125
kubectl wait hr kubernetes-${test_name}-cilium -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-coredns -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-csi -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-ingress-nginx -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-vsnap-crd -n tenant-test --timeout=1m --for=condition=ready

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

These kubectl wait commands for HelmReleases are repetitive. You can use a for loop to make the script more concise and easier to maintain. This will be helpful if you need to add or remove components in the future.

Suggested change
kubectl wait hr kubernetes-${test_name}-cilium -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-coredns -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-csi -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-ingress-nginx -n tenant-test --timeout=1m --for=condition=ready
kubectl wait hr kubernetes-${test_name}-vsnap-crd -n tenant-test --timeout=1m --for=condition=ready
for component in cilium coredns csi ingress-nginx vsnap-crd; do
kubectl wait hr kubernetes-${test_name}-${component} -n tenant-test --timeout=1m --for=condition=ready
done

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 945887f and 626a666.

📒 Files selected for processing (5)
  • hack/e2e-apps/run-kubernetes.sh (1 hunks)
  • packages/apps/kubernetes/Makefile (1 hunks)
  • packages/apps/tenant/values.yaml (1 hunks)
  • packages/core/installer/Makefile (1 hunks)
  • packages/core/installer/values.yaml (0 hunks)
💤 Files with no reviewable changes (1)
  • packages/core/installer/values.yaml
🧰 Additional context used
🪛 checkmake (0.2.2)
packages/apps/kubernetes/Makefile

[warning] 1-1: Missing required phony target "all"

(minphony)


[warning] 1-1: Missing required phony target "clean"

(minphony)


[warning] 1-1: Missing required phony target "test"

(minphony)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build
🔇 Additional comments (3)
packages/core/installer/Makefile (1)

27-28: Explicit amd64 platform is helpful

Locking the build to linux/amd64 avoids accidental arm64 outputs on Apple Silicon hosts; change looks good.

packages/apps/tenant/values.yaml (1)

10-10: Enabling per-tenant Etcd

Turning this on by default aligns the test environment with real deployments; no issues on my side.

packages/apps/kubernetes/Makefile (1)

1-1: Kubernetes version bump looks consistent

All build targets pull from this variable, so updating it here keeps the images in sync. Looks good.

Comment thread hack/e2e-apps/run-kubernetes.sh
@IvanHunters
IvanHunters force-pushed the feat/impruvement-kubernetes-tests branch 2 times, most recently from ca586b9 to ae0d0e8 Compare October 3, 2025 06:06

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
hack/e2e-apps/run-kubernetes.sh (1)

103-115: Fix kubelet version check to accept exact matches.

Line 107’s regex ^"${k8s_version}"[^$]+ insists on an extra character after the version, so nodes reporting the exact semver (e.g., v1.33.2) always fail this test. Switch to a prefix/exact match and break as soon as a mismatch appears.

-  for v in $versions; do
-      if [[ ! "$v" =~ ^"${k8s_version}"[^$]+ ]]; then
-          node_ok=false
-      fi
-  done
+  for v in $versions; do
+      case "$v" in
+          "${k8s_version}"|"${k8s_version}".*|"${k8s_version}"-*)
+              ;;
+          *)
+              node_ok=false
+              break
+              ;;
+      esac
+  done
🧹 Nitpick comments (1)
hack/e2e-apps/run-kubernetes.sh (1)

89-90: Update comment to reflect 150-second timeout.

Line 89 still references 40 seconds, but the command now runs with timeout 150s. Please sync the comment to avoid confusion.

-  # Set up port forwarding to the Kubernetes API server for a 40 second timeout
+  # Set up port forwarding to the Kubernetes API server with a 150-second timeout
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 626a666 and ca586b9.

📒 Files selected for processing (2)
  • hack/e2e-apps/run-kubernetes.sh (1 hunks)
  • packages/apps/kubernetes/Makefile (1 hunks)
🧰 Additional context used
🪛 checkmake (0.2.2)
packages/apps/kubernetes/Makefile

[warning] 1-1: Missing required phony target "all"

(minphony)


[warning] 1-1: Missing required phony target "clean"

(minphony)


[warning] 1-1: Missing required phony target "test"

(minphony)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
@IvanHunters
IvanHunters force-pushed the feat/impruvement-kubernetes-tests branch from ae0d0e8 to 670341f Compare October 3, 2025 06:07
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
hack/e2e-apps/run-kubernetes.sh (1)

89-91: Update the timeout comment.

The comment still says “40 second timeout,” but the command now uses timeout 200s. Please sync the wording to avoid confusion for future readers.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 670341f and a8562f0.

📒 Files selected for processing (1)
  • hack/e2e-apps/run-kubernetes.sh (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build

Comment thread hack/e2e-apps/run-kubernetes.sh
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
hack/e2e-apps/run-kubernetes.sh (1)

89-90: Align the timeout comment with the code.

Line 89 still claims a 40 s timeout, but the command on Line 90 now runs timeout 200s. Please update the comment to avoid confusing future readers.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a8562f0 and 6937b8e.

📒 Files selected for processing (1)
  • hack/e2e-apps/run-kubernetes.sh (1 hunks)

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6937b8e and f2cd585.

📒 Files selected for processing (1)
  • hack/e2e-apps/run-kubernetes.sh (1 hunks)

Comment thread hack/e2e-apps/run-kubernetes.sh Outdated
Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>

@kvaps Andrei Kvapil (kvaps) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kvaps
Andrei Kvapil (kvaps) merged commit dfb0838 into main Oct 6, 2025
59 of 62 checks passed
@kvaps
Andrei Kvapil (kvaps) deleted the feat/impruvement-kubernetes-tests branch October 6, 2025 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants