ci(release): add dispatchable lane to redo finalize registry side effects - #3438
ci(release): add dispatchable lane to redo finalize registry side effects#3438myasnikovdaniil wants to merge 1 commit into
Conversation
…ects The finalize job's registry steps (retag rc images to stable, publish the stable cozy-installer chart) run on the promote-PR merge commit, so a re-run after a mid-flight failure reuses the same tree — a script fix landed on main can never reach it. v1.6.0 hit exactly this: 32 of 43 repos were left untagged and the chart unpublished, recovered by hand with 64 skopeo copies plus helm package/push from the tag tree. Add redo-registry-finalize.yaml, a workflow_dispatch(tag) lane that repeats the registry part for an existing stable tag with a dual checkout: scripts come from the dispatch-time main ref, the release tree (digest pins, chart sources) from refs/tags/<tag>. Guards: strict stable-tag validation, dispatch-from-main check, published-release check, and the same max-published-stable decision as finalize so replaying an old release can repair its immutable tags but never steal :latest. Images stay write-once via promote-retag.sh; the chart step is verify-or-fail — an already-published chart is pulled and compared by extracted file contents, a mismatch fails red before any :latest move unless force_chart=true consciously republishes. Auth is the built-in GITHUB_TOKEN with packages: write only; the lane pushes no git refs. Assisted-By: GPT-5 <noreply@openai.com> Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
📝 WalkthroughWalkthroughA manually triggered workflow validates a stable release tag, recomputes ChangesRegistry finalization recovery
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/redo-registry-finalize.yaml:
- Around line 187-194: Harden the “Set up toolchain (skopeo, yq, helm)” step by
replacing the mutable yq latest download and Helm installer script from main
with pinned released versions and verified checksums, or maintained setup
actions that provide equivalent verification. Preserve installation only when
the tools are unavailable, and ensure both tools are fetched and executed
through a reproducible, integrity-checked path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: f27cf102-ff50-4539-8827-2d29d6b6dd45
📒 Files selected for processing (1)
.github/workflows/redo-registry-finalize.yaml
| - name: Set up toolchain (skopeo, yq, helm) | ||
| run: | | ||
| command -v yq >/dev/null \ | ||
| || { sudo curl -sSL -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq; } | ||
| command -v skopeo >/dev/null \ | ||
| || { sudo apt-get update -qq && sudo apt-get install -y -qq skopeo; } | ||
| command -v helm >/dev/null \ | ||
| || curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Unpinned, unverified tool fetches in a packages: write job.
yq is downloaded from the latest GitHub release asset with no checksum/signature check, and helm is installed by piping get-helm-3 from the mutable main branch straight into bash. Both are moving targets: a compromise of either upstream account/CDN, or simply a breaking upstream change, runs unverified code in a job that holds packages: write to GHCR and pushes images/charts. Pin to a specific released version and verify a checksum (or use a maintained setup action) for both tools.
🔒 Example hardening direction
- name: Set up toolchain (skopeo, yq, helm)
run: |
command -v yq >/dev/null \
- || { sudo curl -sSL -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 && sudo chmod +x /usr/local/bin/yq; }
+ || {
+ YQ_VERSION="v4.44.3"
+ YQ_SHA256="<pin-the-published-checksum-for-this-version>"
+ sudo curl -sSL -o /usr/local/bin/yq "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64"
+ echo "${YQ_SHA256} /usr/local/bin/yq" | sha256sum -c -
+ sudo chmod +x /usr/local/bin/yq
+ }
command -v skopeo >/dev/null \
|| { sudo apt-get update -qq && sudo apt-get install -y -qq skopeo; }
command -v helm >/dev/null \
- || curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
+ || curl -fsSL https://raw.githubusercontent.com/helm/helm/v3.15.4/scripts/get-helm-3 | bash🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/redo-registry-finalize.yaml around lines 187 - 194, Harden
the “Set up toolchain (skopeo, yq, helm)” step by replacing the mutable yq
latest download and Helm installer script from main with pinned released
versions and verified checksums, or maintained setup actions that provide
equivalent verification. Preserve installation only when the tools are
unavailable, and ensure both tools are fetched and executed through a
reproducible, integrity-checked path.
IvanHunters
left a comment
There was a problem hiding this comment.
LGTM — no blockers.
Verified the "fixes from main × release pins from the tag" mechanism via hack/lib/image-refs.sh (root passed as arg), the semver-regex gate on the dispatched tag (no injection sink — everything flows through env vars), and least-privilege permissions (contents: read + packages: write, built-in GITHUB_TOKEN, no contents: write unlike finalize). Image + chart steps are idempotent (write-once + a diff-guard the regular finalize lacks).
Non-blocking notes:
- concurrency.group
redo-registry-finalize(:38) serializes redo runs only among themselves, not against the regular finalize job — worth a comment noting the boundary. - The inline github-script latest-decision logic (:107-161) is untested (dispatch-only, statically unverifiable); same as finalize, not a regression.
- The chart-existence probe
skopeo inspect --raw ... 2>/dev/null(:252) treats any transient/registry error as "not published" → unconditional helm push, bypassing the diff-guard. Harmless in practice but a false-negative path. - The lane only backfills MISSING tags; a tag already pointing at a wrong digest fails by write-once (by design). Making that boundary explicit in a comment would help future operators.
|
Closing in favour of #3456, which fixes the same failure by making the existing finalize registry steps re-runnable rather than adding a second lane. The problem statement here is right — a re-run checks out the merge commit, so a corrected script can never reach it. But that is a property of where the script comes from, not of the trigger. Checking the promotion scripts out from What that avoids is duplication. This lane re-derives the max-semver The coverage the dispatch lane uniquely adds is repairing a release more than 30 days old (GitHub's re-run limit) or recovering from a broken workflow file rather than a broken script. Neither has occurred; the case that did occur is covered. For those, the scripts now run by hand from a checkout of the release tag, which is documented in The chart content comparison from this PR is kept — it moves into |
What this PR does
The finalize job's registry side effects (retag rc images to stable, publish the stable cozy-installer chart) run on the promote-PR merge commit, so a re-run after a mid-flight failure checks out the same tree — a script fix landed on
maincan never reach it. v1.6.0 hit exactly this: the digest-verification bug left 32 of 43 repos untagged and the chart unpublished, and recovery meant 64 manualskopeo copyinvocations plushelm package/helm pushfrom the tag tree.This adds
redo-registry-finalize.yaml, aworkflow_dispatch(tag)lane that repeats the registry part of finalize for an existing stable tag. The core is a dual checkout: scripts (hack/promote-retag.shand its lib) come from the dispatch-timemainref, while the release tree (digest pins, chart sources) comes from a second checkout ofrefs/tags/<tag>— so a corrected script applies to any past tag.Guards: strict
vX.Y.Ztag validation, a dispatched-from-main check, a published-release-exists check, and the same max-published-stable decision as finalize for:latest, so replaying an old release repairs its immutable tags but can never steal the floating tag. Images stay write-once viapromote-retag.sh(skip when identical, hard-fail on digest mismatch). The chart step is verify-or-fail for symmetry: an already-published chart is pulled and compared by extracted file contents (tar/gzip metadata excluded), a mismatch fails red before any:latestmove, and a conscious republish requires theforce_chartinput. Auth is the built-inGITHUB_TOKENwithpackages: writeonly; the lane pushes no git refs, and the whole run is idempotent — an all-done release comes out green.Verification:
actionlintclean;zizmorclean including thepedanticandauditorpersonas; dry-traced against the v1.6.0 incident state (32 missing tags + unpublished chart converge to green) and against a hypothetical mis-published chart (red withoutforce_chart, republish with it).Release note
Summary by CodeRabbit