Skip to content

test(e2e): widen etcd convergence budget to 5m in Talos bootstrap - #3036

Merged
IvanHunters merged 1 commit into
mainfrom
ray/e2e-bootstrap-etcd-timeout
Jun 25, 2026
Merged

test(e2e): widen etcd convergence budget to 5m in Talos bootstrap#3036
IvanHunters merged 1 commit into
mainfrom
ray/e2e-bootstrap-etcd-timeout

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Problem

The Bootstrap Talos cluster E2E step fails intermittently with ❌ Test failed: Bootstrap Talos cluster (exit 1) even though the cluster is healthy. The exit comes from the etcd-members convergence gate, which is bounded at timeout 180.

A 3-node etcd converges via Talos's serialized learner promotion — etcd admits one learner at a time, so the third member is only promoted after the second has fully caught up. On a loaded ephemeral runner this deterministic path can legitimately approach ~3 minutes of wall clock and photo-finish against the 180s ceiling, timing out a cluster that has in fact converged (every member reaches Health check successful; no kernel panic / OOM / boot failure / network-unreachable in dmesg). Observed: a run where the third member reached etcd health at +178.6s against the 180s budget, failing ~1.4s short.

Fix

Widen the etcd-members convergence budget from 180s to 300s. The wait is already event-driven (until talosctl etcd members …; do sleep 1; done) — only the ceiling was too tight. No retry is added and the dmesg-on-failure diagnostic is preserved.

300s matches the repo's other in-line budgets for deterministic convergence (HelmRelease-ready 5m, harbor/NFS 10m, install 15m).

Why not just rerun

This is not environmental noise: the cluster converges correctly every time, it is only timed out. Rerunning wastes a full ~1h E2E cycle to re-roll the dice on whether the runner shaves the last ~2s. Correcting the mistuned budget removes the photo-finish.

Summary by CodeRabbit

  • Bug Fixes
    • Extended the cluster bootstrap wait time for control plane health checks, reducing false failures during slow startup.
    • Improved readiness handling so cluster setup is more resilient before moving on to later initialization steps.

The "Bootstrap Talos cluster" step times out the etcd-members health gate
at 180s. A 3-node etcd converges via Talos's serialized learner promotion
(one learner admitted at a time), so on a loaded ephemeral runner this
deterministic path can approach ~3m wall clock and photo-finish against the
180s ceiling — failing a cluster that is in fact healthy. The wait is
already event-driven; only the ceiling was too tight. Widen it to 300s.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@github-actions github-actions Bot added the size/S This PR changes 10-29 lines, ignoring generated files label Jun 24, 2026
@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23aa3558-58ed-47e6-b79a-4d22b1fc58a1

📥 Commits

Reviewing files that changed from the base of the PR and between 29bf56f and c7bd3bc.

📒 Files selected for processing (1)
  • hack/e2e-prepare-cluster.bats

📝 Walkthrough

Walkthrough

The Talos cluster bootstrap script now waits longer for etcd readiness before failing, and its inline comment was updated. The later logging and exit behavior after the check remain unchanged.

Changes

Talos cluster bootstrap wait

Layer / File(s) Summary
etcd readiness wait
hack/e2e-prepare-cluster.bats
The etcd polling timeout during bootstrap was increased from 180s to 300s, and the adjacent comment was rewritten; the post-check error handling stays the same.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Poem

I hopped through bootstrap, soft and slow,
Five minutes now lets etcd grow.
A tiny nudge, a patient beat,
Then nodes appear on steady feet. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: extending the etcd convergence timeout in the Talos E2E bootstrap test.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ray/e2e-bootstrap-etcd-timeout

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses intermittent E2E test failures occurring during the Talos cluster bootstrap phase. By extending the timeout for etcd member convergence, the change accommodates the natural, serialized promotion process of etcd members, which can occasionally exceed the previous 3-minute limit under high system load without indicating an actual cluster health issue.

Highlights

  • Increased etcd convergence timeout: Updated the timeout for the etcd-members convergence check in the Talos bootstrap process from 180 seconds to 300 seconds to prevent intermittent failures on loaded ephemeral runners.
  • Improved documentation: Added detailed comments explaining why the 5-minute budget is necessary due to Talos's serialized learner promotion mechanism.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request increases the timeout for waiting until etcd is healthy from 180 to 300 seconds in hack/e2e-prepare-cluster.bats to prevent premature timeouts on loaded ephemeral runners. The review feedback notes that the PR title uses an invalid scope (e2e instead of tests or hack) and still contains a placeholder release note, violating the repository's commit and PR guidelines.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +264 to +265
# Wait until etcd is healthy.
# Budget is 5m, not 3m: a 3-node etcd converges via Talos's serialized

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The pull request does not fully comply with the repository's commit and PR requirements:

  1. Invalid Scope in PR Title/Commit: The PR title uses the scope e2e (test(e2e): ...). According to the Repository Style Guide (lines 40-46), e2e is not a valid scope. Please update the scope to a valid one, such as tests or hack (e.g., test(tests): ... or test(hack): ...).
  2. Placeholder Release Note: The release-note block in the PR body is still using the default placeholder template (type(scope): human-readable changelog entry). Please update it with a proper human-readable changelog entry describing this change.
References
  1. Commit and PR Requirements: Each commit must follow Conventional Commits format with a valid scope (e.g., tests or hack instead of e2e). The PR body must contain a completed release-note block rather than the default placeholder. (link)

@IvanHunters IvanHunters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the diff. Timeout-only change in hack/e2e-prepare-cluster.bats, no prod path touched; 5m budget aligned with existing HelmRelease/install timeouts. LGTM.

@IvanHunters
IvanHunters merged commit 144f592 into main Jun 25, 2026
12 checks passed
@IvanHunters
IvanHunters deleted the ray/e2e-bootstrap-etcd-timeout branch June 25, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/testing Issues or PRs related to testing (e2e, bats, unit tests) size/S This PR changes 10-29 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants