Skip to content

fix(platform): make etcd v1alpha2 adoption (migration 50) robust in-cluster - #3261

Closed
Andrey Kolkov (androndo) wants to merge 2 commits into
mainfrom
fix/etcd-migration50-adoption-robustness
Closed

fix(platform): make etcd v1alpha2 adoption (migration 50) robust in-cluster#3261
Andrey Kolkov (androndo) wants to merge 2 commits into
mainfrom
fix/etcd-migration50-adoption-robustness

Conversation

@androndo

@androndo Andrey Kolkov (androndo) commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Migration 50 (etcd.aenix.io -> etcd-operator.cozystack.io/v1alpha2 adoption) had two defects that blocked every in-cluster 1.5 -> 1.6 upgrade on a cluster with an existing etcd:

  1. Cert-SAN wait treated transient kubectl failures as "SAN absent". ensure_wildcard_sans checked/awaited the wildcard SAN with kubectl get ... 2>/dev/null | grep, so any transient GET failure (API discovery refresh, apiserver blip, throttling) produced an empty string indistinguishable from a genuine absence -> false miss, and the 120s wait never recovered. Replace the two ad-hoc checks with a _san_present helper that retries on an empty read (a real Certificate/Secret never has empty dnsNames/alt-names) and accepts the native wildcard from EITHER the issued Secret's cert-manager.io/alt-names annotation OR the Certificate spec.dnsNames (the source of truth for what cert-manager will issue).

  2. etcd-migrate had no kubeconfig in-cluster. etcd-migrate only reads a kubeconfig file (-k/--kubeconfig, default /root/.kube/config) and, unlike kubectl, does not fall back to the mounted in-cluster ServiceAccount. The hook Job set no KUBECONFIG and passed no --kubeconfig, so both the dry-run and --apply aborted with "error building kubeconfig: stat /root/.kube/config: no such file". Synthesize an in-cluster kubeconfig from the mounted ServiceAccount and pass --kubeconfig to both etcd-migrate invocations.

Verified end-to-end on a 1.5.2 -> 1.6.0-rc.1 upgrade: the adoption now completes in-place (pods never restarted, data intact) and the cluster reaches readyMembers=3 / Available=True.

Refs: #3243, #3255

What this PR does

Screenshots

Release note


Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of wildcard SAN detection by validating against both cert-manager sources and retrying when reads are temporarily empty.
    • Updated Secret-based SAN presence checks to retry on missing annotation data and require an exact match for the native wildcard.
    • Fixed migration authentication by generating a kubeconfig for etcd-migrate from the hook Pod’s ServiceAccount credentials, including during dry-run and apply.
  • Tests
    • Added end-to-end coverage for exact wildcard SAN matching behavior and kubeconfig generation/authentication.

…luster

Migration 50 (etcd.aenix.io -> etcd-operator.cozystack.io/v1alpha2 adoption)
had two defects that blocked every in-cluster 1.5 -> 1.6 upgrade on a cluster
with an existing etcd:

1. Cert-SAN wait treated transient kubectl failures as "SAN absent".
   ensure_wildcard_sans checked/awaited the wildcard SAN with
   `kubectl get ... 2>/dev/null | grep`, so any transient GET failure (API
   discovery refresh, apiserver blip, throttling) produced an empty string
   indistinguishable from a genuine absence -> false miss, and the 120s wait
   never recovered. Replace the two ad-hoc checks with a _san_present helper
   that retries on an empty read (a real Certificate/Secret never has empty
   dnsNames/alt-names) and accepts the native wildcard from EITHER the issued
   Secret's cert-manager.io/alt-names annotation OR the Certificate
   spec.dnsNames (the source of truth for what cert-manager will issue).

2. etcd-migrate had no kubeconfig in-cluster.
   etcd-migrate only reads a kubeconfig file (-k/--kubeconfig, default
   /root/.kube/config) and, unlike kubectl, does not fall back to the mounted
   in-cluster ServiceAccount. The hook Job set no KUBECONFIG and passed no
   --kubeconfig, so both the dry-run and --apply aborted with
   "error building kubeconfig: stat /root/.kube/config: no such file".
   Synthesize an in-cluster kubeconfig from the mounted ServiceAccount and pass
   --kubeconfig to both etcd-migrate invocations.

Verified end-to-end on a 1.5.2 -> 1.6.0-rc.1 upgrade: the adoption now
completes in-place (pods never restarted, data intact) and the cluster reaches
readyMembers=3 / Available=True.

Refs: #3243, #3255

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request improves the reliability of the etcd v1alpha2 migration process (migration 50) for in-cluster upgrades. By addressing transient API errors during certificate validation and ensuring proper authentication for the migration tool, the changes ensure that the migration process is more resilient to environmental blips and correctly operates within the Kubernetes service account context.

Highlights

  • Robustness in Certificate SAN Verification: Introduced a new _san_present helper function to replace ad-hoc checks. This helper adds retry logic to handle transient API failures during certificate SAN verification, preventing false negatives that previously caused migration timeouts.
  • In-Cluster Kubeconfig Synthesis: Implemented dynamic generation of a kubeconfig file from the mounted ServiceAccount token. This ensures etcd-migrate can authenticate correctly in-cluster, resolving previous execution failures due to missing configuration files.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@dosubot dosubot Bot added area/platform Issues or PRs related to platform infrastructure (bundle, flux, talos, installer) backport Should change be backported on previous release kind/bug Categorizes issue or PR as related to a bug priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now labels Jul 9, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces robustness improvements to the migration script in packages/core/platform/images/migrations/migrations/50. It adds a _san_present helper function with retries to handle transient API hiccups when checking for SAN presence, and synthesizes an in-cluster kubeconfig file for etcd-migrate to authenticate correctly. The review feedback identifies two key issues: a potential false positive in SAN matching due to substring matching on spec.dnsNames, and a potential failure on IPv6-only or dual-stack clusters when constructing the Kubernetes API server URL, both of which have actionable code suggestions to resolve them.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +132 to +136
d=$(kubectl -n "$ns" get certificate.cert-manager.io "$cert" \
-o jsonpath='{.spec.dnsNames}' 2>/dev/null || true)
if [ -n "$a" ] || [ -n "$d" ]; then
printf '%s' "$a" | tr ',' '\n' | grep -qxF "$native" && return 0
printf '%s' "$d" | tr ',' '\n' | grep -qF "$native" && return 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using {.spec.dnsNames} with tr ',' '\n' and grep -qF (substring match) can lead to false positives if the target wildcard SAN is a substring of another SAN (e.g., *.etcd.tenant-foo.svc matching *.etcd.tenant-foo.svc.cluster.local).

Instead, we can use {.spec.dnsNames[*]} to get a space-separated list of DNS names, and then use tr ' ' '\n' with grep -qxF for a robust, exact line match.

Suggested change
d=$(kubectl -n "$ns" get certificate.cert-manager.io "$cert" \
-o jsonpath='{.spec.dnsNames}' 2>/dev/null || true)
if [ -n "$a" ] || [ -n "$d" ]; then
printf '%s' "$a" | tr ',' '\n' | grep -qxF "$native" && return 0
printf '%s' "$d" | tr ',' '\n' | grep -qF "$native" && return 0
d=$(kubectl -n "$ns" get certificate.cert-manager.io "$cert" \
-o jsonpath='{.spec.dnsNames[*]}' 2>/dev/null || true)
if [ -n "$a" ] || [ -n "$d" ]; then
printf '%s' "$a" | tr ',' '\n' | grep -qxF "$native" && return 0
printf '%s' "$d" | tr ' ' '\n' | grep -qxF "$native" && return 0

- name: in-cluster
cluster:
certificate-authority: ${_sa_dir}/ca.crt
server: https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT:-443}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT:-443} can fail on IPv6-only or dual-stack clusters because raw IPv6 addresses in URLs must be enclosed in square brackets (e.g., https://[2001:db8::1]:443).

Since this migration hook runs inside the cluster where DNS is active, we can use the standard, built-in DNS name https://kubernetes.default.svc instead. This is fully robust and works seamlessly across both IPv4 and IPv6 environments.

Suggested change
server: https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT:-443}
server: https://kubernetes.default.svc

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 635c2c7c-6925-4ab0-abad-bfe76256ce86

📥 Commits

Reviewing files that changed from the base of the PR and between b89b159 and b43abb2.

📒 Files selected for processing (3)
  • hack/migration-50-etcd-adopt.bats
  • hack/testdata/migration-50/kubectl
  • packages/core/platform/images/migrations/migrations/50

📝 Walkthrough

Walkthrough

Migration 50 now retries wildcard SAN detection across cert-manager sources and uses a synthesized in-cluster kubeconfig for both etcd-migrate runs. Tests and fake kubectl output were updated to cover the new SAN matching and kubeconfig behavior.

Changes

Migration script 50 fixes

Layer / File(s) Summary
Wildcard SAN presence check
packages/core/platform/images/migrations/migrations/50, hack/testdata/migration-50/kubectl, hack/migration-50-etcd-adopt.bats
Adds retrying SAN helpers that inspect both Secret annotations and Certificate dnsNames, wires exact-match detection into the initial and post-patch checks, and extends test/fake kubectl coverage for the substring SAN case.
etcd-migrate kubeconfig synthesis
packages/core/platform/images/migrations/migrations/50, hack/migration-50-etcd-adopt.bats
Generates /tmp/in-cluster.kubeconfig from the mounted ServiceAccount files, passes it to dry-run and apply etcd-migrate calls, and verifies the produced kubeconfig contents in tests.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

Suggested labels: area/testing

Suggested reviewers: kvaps, sircthulhu, lllamnyp, myasnikovdaniil

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: hardening migration 50 for in-cluster etcd v1alpha2 adoption.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/etcd-migration50-adoption-robustness

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/core/platform/images/migrations/migrations/50 (1)

175-182: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Wait on the Secret, not the Certificate spec

_san_present returns success from Certificate.spec.dnsNames, and this loop runs right after that field is patched. That makes the first iteration succeed immediately, so migration can continue before cert-manager has re-issued the Secret. Check only cert-manager.io/alt-names here.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/platform/images/migrations/migrations/50` around lines 175 -
182, The polling loop around _san_present is checking Certificate.spec.dnsNames,
which is updated immediately by the patch and lets the migration continue too
early. Update the logic in this loop to wait only for cert-manager.io/alt-names
on the Secret, using _san_present as the call site reference but removing the
Certificate.spec.dnsNames dependency so success means the Secret has actually
been re-issued.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@packages/core/platform/images/migrations/migrations/50`:
- Around line 175-182: The polling loop around _san_present is checking
Certificate.spec.dnsNames, which is updated immediately by the patch and lets
the migration continue too early. Update the logic in this loop to wait only for
cert-manager.io/alt-names on the Secret, using _san_present as the call site
reference but removing the Certificate.spec.dnsNames dependency so success means
the Secret has actually been re-issued.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 53bd6a1c-3dfc-4613-950e-77d5cb9302b9

📥 Commits

Reviewing files that changed from the base of the PR and between f91841f and b89b159.

📒 Files selected for processing (1)
  • packages/core/platform/images/migrations/migrations/50

@myasnikovdaniil myasnikovdaniil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOT LGTM — both fixes (#3243 cert-SAN wait, #3255 in-cluster kubeconfig) are correct and pass the existing suite, but they ship untested in a bats suite that exists specifically to pin this file's contract.

Business context: Migration 50 (adopt legacy etcd.aenix.io/v1alpha1 clusters onto etcd-operator.cozystack.io/v1alpha2) hard-fails every in-cluster 1.5→1.6 upgrade on a cluster with an existing etcd; this PR fixes the two documented blockers.

I verified the happy path: all 11 existing tests in hack/migration-50-etcd-adopt.bats still pass on this branch — no regression.

Blockers

B1: Both fixes ship with no test coverage in a suite built to pin this file's contract

hack/migration-50-etcd-adopt.bats drives the real script against a fake kubectl/etcd-migrate, and its header states the behaviours it pins "are the review blockers." This PR adds _san_present and the in-cluster kubeconfig synthesis (+ --kubeconfig on both etcd-migrate calls) and updates none of it.

Evidence:

  • migrations/50 lines 389 and 435 pass --kubeconfig="${ETCD_MIGRATE_KUBECONFIG}" to the dry-run and --apply. The fake etcd-migrate already logs its full argument vector, yet nothing in the test asserts the flag (grep -c kubeconfig on the .bats = 0). #3255 was a total upgrade blocker that shipped once precisely because no test asserted etcd-migrate is invoked with working auth — without an assertion, a future refactor can silently drop --kubeconfig and re-introduce it. This is ~2 lines: assert --kubeconfig= on the ETCD-MIGRATE cmdlog entries.
  • The _san_present transient-retry path (the core of the #3243 fix) is never exercised: the fake always returns a non-empty alt-names, so the for try in 1 2 3 4 5 empty-read retry never iterates. A fake knob that returns empty N times then succeeds would pin it.
  • The existing "re-issues … certs" test passes only incidentally — the fake serves get certificate … -o jsonpath='{.spec.dnsNames}' as empty, so the new d branch is dead in-test and the assertion still rides the old Secret-annotation path. The new logic is genuinely uncovered.

At minimum add the --kubeconfig assertion; ideally the transient-retry case too.

Non-blocking follow-ups

  1. The cert re-issue wait no longer waits for the Secret; the comment and echo now misstate what happens. Detail inline on line 176. In short: _san_present returns as soon as the Certificate spec.dnsNames carries the wildcard, but the patch a few lines above just added it to spec.dnsNames — so the wait breaks on the first iteration, before cert-manager re-issues the Secret. The comment on line 173 ("Wait for cert-manager to re-issue the Secret") and the echo on line 177 ("re-issued with native wildcard") then describe something that has not been confirmed. Happy path is fine (re-issue takes seconds, member replacement is far later), but the Secret-level confirmation this block was written to provide is gone. Either keep the wait checking the Secret only, or — if trusting the patched spec is intentional — drop the now-dead 120s loop and correct the comment/echo.

  2. Substring vs exact SAN match in _san_present — inline on line 136.

  3. IPv6-only server URL in the synthesized kubeconfig — inline on line 371.

  4. Nit: the release-note block is empty for a change that unblocks every 1.5→1.6 upgrade — worth a line.

Items 2 and 3 overlap with the automated review already on this PR; I verified both independently and they hold.

-o jsonpath='{.spec.dnsNames}' 2>/dev/null || true)
if [ -n "$a" ] || [ -n "$d" ]; then
printf '%s' "$a" | tr ',' '\n' | grep -qxF "$native" && return 0
printf '%s' "$d" | tr ',' '\n' | grep -qF "$native" && return 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_san_present matches the Secret annotation with grep -qxF (exact line, L135) but the Certificate dnsNames with grep -qF (substring, here). The substring form is forced by reading {.spec.dnsNames}, which renders as a bracketed blob [a b c] that tr ',' '\n' cannot split. A wildcard that is a substring of a longer SAN (e.g. *.etcd.<ns>.svc inside *.etcd.<ns>.svc.cluster.local) would false-positive and skip the patch. This chart only issues the short .svc forms, so no live bug today — but it is fragile. Reading {.spec.dnsNames[*]} (space-separated) and matching with tr ' ' '\n' | grep -qxF makes both branches exact.

if kubectl -n "$ns" get secret "$secret" \
-o jsonpath='{.metadata.annotations.cert-manager\.io/alt-names}' 2>/dev/null \
| tr ',' '\n' | grep -qxF "$native"; then
if _san_present "$ns" "$secret" "$cert" "$native"; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the wait-for-re-issue loop, but _san_present returns as soon as the Certificate spec.dnsNames contains the wildcard — and the patch a few lines above just added it to spec.dnsNames. So this breaks on the first iteration, before cert-manager re-issues the Secret. The comment on L173 and the echo on L177 ("re-issued with native wildcard") then print during an unattended upgrade without the Secret actually being confirmed. On the happy path cert-manager re-issues within seconds and member replacement happens far later, so it works — but the Secret-level guarantee this block was written for is lost. Either scope the wait-loop check to the Secret annotation only, or, if trusting the patched spec is intentional, remove the now-dead 30×4s loop and fix the comment + echo.

- name: in-cluster
cluster:
certificate-authority: ${_sa_dir}/ca.crt
server: https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT:-443}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On an IPv6-only cluster KUBERNETES_SERVICE_HOST is a bare IPv6 address, so this produces an invalid URL (https://fd00::1:443 — the host must be bracketed). Dual-stack uses the IPv4 primary, so the common path is unaffected, but server: https://kubernetes.default.svc is IP-family-agnostic, validated by the mounted SA CA, and matches how kubectl builds its own in-cluster server URL.

@github-actions github-actions Bot added the size/M This PR changes 30-99 lines, ignoring generated files label Jul 9, 2026
…t-gated wait, IPv6 kubeconfig)

Review feedback on #3261 (gemini-code-assist, coderabbitai, myasnikovdaniil):

- Exact SAN match. _san_present read the Certificate SANs as {.spec.dnsNames}
  (a bracketed JSON blob that `tr ','` cannot split) and matched with a
  substring `grep -qF`, so a wildcard that is a substring of a longer SAN
  (e.g. *.etcd.<ns>.svc inside *.etcd.<ns>.svc.cluster.local) could
  false-positive and skip the re-issue. Read {.spec.dnsNames[*]} (space
  separated) and match exactly with `grep -qxF`, matching the Secret-annotation
  branch.

- Wait on the re-issued Secret, not the patched spec. The post-patch wait loop
  called _san_present, which returns as soon as the Certificate spec.dnsNames
  contains the wildcard -- but the patch just added it there, so the loop broke
  on the first iteration before cert-manager re-issued the Secret, and the
  "re-issued" log printed unconfirmed. Add _secret_has_san (Secret alt-names
  annotation only, same empty-read retry + exact match) and gate the wait on it.

- IPv6-safe kubeconfig server. The synthesized kubeconfig used
  https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT, which is an invalid
  URL on an IPv6-only cluster (a bare IPv6 host must be bracketed). Use
  https://kubernetes.default.svc -- IP-family-agnostic, validated by the mounted
  SA CA, and the same server URL kubectl synthesizes in-cluster.

Tests (the review blocker): extend the bats suite (and its fake kubectl) with
two cases pinning these contracts -- a superstring dnsName must NOT skip the
re-issue patch, and etcd-migrate must be invoked with a synthesized in-cluster
kubeconfig whose server is kubernetes.default.svc (not the bare IPv6 host),
authenticating via the SA token file + CA. Both the kubeconfig path and the SA
directory are now env-overridable (ETCD_MIGRATE_KUBECONFIG, ETCD_ADOPT_SA_DIR)
so the synthesis is exercisable off-cluster; production defaults are unchanged.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files and removed size/M This PR changes 30-99 lines, ignoring generated files labels Jul 9, 2026

@myasnikovdaniil myasnikovdaniil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the follow-up commit resolves all four points; both fixes are correct, now pinned by tests, and the full migration-50 suite is green (13/13).

Verified on b43abb2a:

  • Exact SAN match_san_present now reads {.spec.dnsNames[*]} and matches with grep -qxF, consistent with the Secret-annotation branch. Pinned by the new "superstring dnsName does not skip the re-issue patch" test (confirmed: reverting it to grep -qF makes that test fail).
  • Secret-gated wait — the re-issue wait now uses _secret_has_san, which checks only the issued Secret's alt-names (not the just-patched spec) while keeping the transient-empty-read retry; the echo is corrected. This restores the Secret-level confirmation the block exists for.
  • IPv6 kubeconfigserver: https://kubernetes.default.svc, validated by the mounted SA CA and IP-family-agnostic. Pinned by the new kubeconfig test (asserts --kubeconfig on both etcd-migrate calls and that the bare IPv6 host never lands in the URL; confirmed dropping a --kubeconfig makes it fail).

ETCD_ADOPT_SA_DIR / ETCD_MIGRATE_KUBECONFIG are test-only seams with unchanged production defaults.

Non-blocking nit (unchanged): the release-note block is still empty for a change that unblocks every 1.5→1.6 upgrade — worth a line for the changelog.

myasnikovdaniil added a commit that referenced this pull request Jul 10, 2026
…sition fixes

Brings the etcd-migrate migration-50 fixes (cert-SAN retry + Secret-gated
wait, in-cluster kubeconfig) alongside the etcd-operator/chart transition
fixes so the full 1.5->1.6 etcd v1alpha2 adoption path lands together.
myasnikovdaniil pushed a commit that referenced this pull request Jul 15, 2026
…t-gated wait, IPv6 kubeconfig)

Review feedback on #3261 (gemini-code-assist, coderabbitai, myasnikovdaniil):

- Exact SAN match. _san_present read the Certificate SANs as {.spec.dnsNames}
  (a bracketed JSON blob that `tr ','` cannot split) and matched with a
  substring `grep -qF`, so a wildcard that is a substring of a longer SAN
  (e.g. *.etcd.<ns>.svc inside *.etcd.<ns>.svc.cluster.local) could
  false-positive and skip the re-issue. Read {.spec.dnsNames[*]} (space
  separated) and match exactly with `grep -qxF`, matching the Secret-annotation
  branch.

- Wait on the re-issued Secret, not the patched spec. The post-patch wait loop
  called _san_present, which returns as soon as the Certificate spec.dnsNames
  contains the wildcard -- but the patch just added it there, so the loop broke
  on the first iteration before cert-manager re-issued the Secret, and the
  "re-issued" log printed unconfirmed. Add _secret_has_san (Secret alt-names
  annotation only, same empty-read retry + exact match) and gate the wait on it.

- IPv6-safe kubeconfig server. The synthesized kubeconfig used
  https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT, which is an invalid
  URL on an IPv6-only cluster (a bare IPv6 host must be bracketed). Use
  https://kubernetes.default.svc -- IP-family-agnostic, validated by the mounted
  SA CA, and the same server URL kubectl synthesizes in-cluster.

Tests (the review blocker): extend the bats suite (and its fake kubectl) with
two cases pinning these contracts -- a superstring dnsName must NOT skip the
re-issue patch, and etcd-migrate must be invoked with a synthesized in-cluster
kubeconfig whose server is kubernetes.default.svc (not the bare IPv6 host),
authenticating via the SA token file + CA. Both the kubeconfig path and the SA
directory are now env-overridable (ETCD_MIGRATE_KUBECONFIG, ETCD_ADOPT_SA_DIR)
so the synthesis is exercisable off-cluster; production defaults are unchanged.

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
myasnikovdaniil added a commit that referenced this pull request Jul 16, 2026
…es (#3270)

Consolidates the etcd `v1alpha2` transition fix for **in-cluster 1.5 →
1.6 upgrades** into a single PR, rebased on current `main`. Supersedes
#3265 and #3261 — their commits are carried here (authorship preserved),
so those PRs can be closed once this lands. Fresh installs use the new
shapes directly and are unaffected, so fresh-install CI does not catch
these; the 1.5 → 1.6 e2e upgrade path is the authoritative regression
guard.

## What this PR does

### 1. Keep the legacy `etcd-headless` Service alive during adoption —
`packages/extra/etcd`
`etcd-migrate` adopts legacy clusters **in place**: the Pods keep their
original `spec.subdomain: etcd-headless` and are dialed at
`etcd-<i>.etcd-headless.<ns>.svc` until they roll onto the operator's
native `<member>.etcd.<ns>.svc` domain. The v1alpha2 operator only
creates the native `etcd` Service and the legacy `etcd-headless` Service
is pruned, so those per-pod names stop resolving (`no such host`),
`MemberList` fails, and `status.readyMembers` never populates — the
`EtcdCluster` never goes `Ready` even though etcd is healthy and in
quorum. We ship a chart-managed transitional headless `etcd-headless`
Service (selector mirrors the operator's native `etcd` Service via
`etcd-operator.cozystack.io/cluster`, `publishNotReadyAddresses: true`)
— the DNS counterpart of the legacy `*.etcd-headless.<ns>.svc` SAN
already kept for this window. Removable together with that SAN once
members roll onto the native subdomain.

### 2. Survive the immutable controller-Deployment selector on upgrade —
`packages/system/etcd-operator` (#3242)
#2859 replaced the upstream etcd-operator chart with the
cozystack-authored one, changing `Deployment.spec.selector.matchLabels`.
`spec.selector` is immutable, so `helm upgrade` cannot patch the
existing Deployment and the whole HelmRelease upgrade fails (`field is
immutable`). A **pre-upgrade hook**
(`templates/pre-upgrade-selector-fix.yaml`: ServiceAccount + Role +
RoleBinding + Job) deletes the Deployment **only** when its live
selector is the pre-1.6 one, so Helm recreates it cleanly. No-op when
the selector already matches, never runs on fresh install.

### 3. Raise the operator's memory cold-start floor —
`packages/system/etcd-operator`
Steady-state working set is ~250Mi; the static `limits.memory: 128Mi`
OOMKills a Pod that starts before the VPA admission webhook rewrites it.
Raise the floor to `256Mi` (and VPA `minAllowed` to match) as defense in
depth so the operator never depends on VPA timing to avoid crashing.

### 4. Make migration 50 (etcd adoption) robust in-cluster —
`packages/core/platform/images/migrations/migrations/50` (was #3261)
Exact server/peer cert-SAN match, Secret-gated wait on the adoption
Secret, and in-cluster (IPv6-safe) kubeconfig handling, so the migration
script drives the adoption reliably from inside the cluster. Covered by
`hack/migration-50-etcd-adopt.bats`.

### 5. Hardening / review fixes (this PR's original scope)
- **Hook `runAsUser: 65532`.** `pre-upgrade-selector-fix.yaml` set
`runAsNonRoot: true` but no numeric `runAsUser`; `clastix/kubectl`'s
image user is the non-numeric name `nonroot`, which the kubelet cannot
verify against `runAsNonRoot` — so the hook Pod fails admission and
silently blocks the very upgrade it exists to unblock. Adds `runAsUser:
65532`, matching the postgres-operator webhook-ready hook that runs the
same image.
- **Digest-pin the kubectl image.** The values comment claimed
digest-pinning but shipped a floating `v1.32` tag. Reuses the digest
postgres-operator vendors, adds the `renovate` annotation, and templates
`repo:tag@digest`.
- **Tests.** `etcd-operator/tests/selector-fix-hook_test.yaml` (hook
wiring, weight ordering, namespaced least-privilege RBAC,
numeric-non-root security context, digest-pinned image),
`etcd-operator/tests/deployment_test.yaml` (256Mi cold-start floor),
`extra/etcd/tests/etcd-cluster_test.yaml` (transitional `etcd-headless`
Service). Each assertion was mutation-tested.

### 6. Derive the default S3 endpoint from the provisioned bucket —
`packages/system/backupstrategy-controller`
Derive the default S3 endpoint (and per-driver scheme / TLS /
`secure_connection`) from the provisioned bucket Secret instead of
requiring it to be hand-set, so etcd (and other) backup strategies get a
working endpoint by default. Docs in
`docs/operations/backup-classes.md`; covered by
`tests/endpoint_form_test.yaml`.

### Verification
- `helm unittest` green on current `main`: etcd-operator **18/18**,
extra/etcd **18/18**, backupstrategy-controller **11/11**.
- Behaviours 1 & 2 were reproduced and confirmed live on a 1.5.2 →
1.6.0-rc.1 adoption (3-node cluster) — recreating the `etcd-headless`
Service took the adopted cluster to `readyMembers=3 / Available=True`.

```release-note
fix(etcd): complete the v1alpha2 transition on in-cluster 1.5→1.6 upgrades — keep the legacy etcd-headless Service alive so adopted members stay resolvable, delete the pre-1.6 operator Deployment via a pre-upgrade hook to get past the immutable selector, raise the operator's memory floor so it does not OOM before the VPA scales it, and make the etcd adoption migration robust in-cluster.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Improved backup storage endpoint handling across supported backup
drivers, including provisioned and external S3 storage.
* Added compatibility support for legacy etcd pod discovery during
migration.
  * Added an automated upgrade safeguard for etcd operator deployments.

* **Bug Fixes**
* Improved certificate SAN detection and etcd migration authentication.
* Increased the etcd operator’s minimum startup memory to prevent early
restarts.

* **Documentation**
  * Clarified backup endpoint, TLS, and driver-specific behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@myasnikovdaniil

Copy link
Copy Markdown
Contributor

Completed in #3270

auto-merge was automatically disabled July 21, 2026 07:55

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/platform Issues or PRs related to platform infrastructure (bundle, flux, talos, installer) backport Should change be backported on previous release kind/bug Categorizes issue or PR as related to a bug priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants