Skip to content

fix(ci): only overlay current-main images on main-based PRs - #3472

Closed
myasnikovdaniil wants to merge 1 commit into
release-1.6from
ci/overlay-guard-release-1.6
Closed

fix(ci): only overlay current-main images on main-based PRs#3472
myasnikovdaniil wants to merge 1 commit into
release-1.6from
ci/overlay-guard-release-1.6

Conversation

@myasnikovdaniil

Copy link
Copy Markdown
Contributor

What this PR does

Backport of #3471 to release-1.6 (clean cherry-pick, -x reference in the commit).

The overlay must be fixed on this branch to have any effect here: for pull_request events GitHub builds the workflow from the merge ref, so a release-line PR only stops receiving main's images once the guard is on the base branch. Landing #3471 on main alone does nothing for the 1.6 line.

Why it matters here specifically

The finalize job pulls cozystack-packages:main and repoints every package a PR did not rebuild at current-main images. On this branch that deploys main's binaries onto release-1.6's charts, and the mismatch grows with every commit main gains.

#3437 — a one-line change deactivating an app in a bundle — failed install deterministically, twice:

helmrelease/backupstrategy-controller: Helm install failed …
error validating data: SchemaError(github.com/cozystack/cozystack/pkg/apis/core/v1alpha1.Option.spec):
unknown model in reference: "github.com~1cozystack~1cozystack~1pkg~1apis~1core~1v1alpha1.OptionSpec"

cozystack-controller:main served an aggregated OpenAPI this branch's charts could not validate against. Every 1.6 backport is exposed to the same thing, and the symptom reads as a broken PR rather than a broken lane.

After this merges, re-running #3437's E2E picks the guard up with no rebase.

Verification

hack/overlay-main-images_test.bats pins the wiring per step, mutation-checked by removing one guard. 12/12 green on this branch, actionlint clean.

Release note

NONE

The PR finalize job pulls cozystack-packages:main and repoints every package
the PR did not rebuild at current-main images. On a main-based PR that is the
whole point: it stops e2e from testing last-release images for everything
outside the build matrix. On a release-line PR it is wrong — the committed refs
there are not stale, they are that line`s released digests, and the charts are
written against exactly those.

So a release-line PR was installing main`s binaries onto its own charts, and the
mismatch grows with every commit main gains. #3437 is the demonstration: a
one-line change on release-1.6 that deactivates an app failed install
deterministically, twice, with

  SchemaError(...core/v1alpha1.Option.spec): unknown model in reference:
  "...core~1v1alpha1.OptionSpec"

from main`s cozystack-controller serving an aggregated OpenAPI that branch`s
charts cannot validate against. Nothing in the PR was broken; the lane was.
Left alone this makes every 1.6 backport look red, which is when release-branch
PRs are busiest.

Both overlay steps are now gated on `github.base_ref == main`, so a release-line
PR keeps its committed digests — the behaviour that predates the overlay.
Retargeting the overlay at a per-line artifact would be better but is not
possible today: build-main.yaml publishes only cozystack-packages:main, and the
registry carries no release-* equivalent (verified against the packages repo`s
tag list: `main` plus per-PR tags, nothing else).

hack/overlay-main-images_test.bats pins the wiring per step, so adding a third
overlay step without the guard fails the suite. Mutation-checked by removing one
guard. 12/12 green; actionlint clean.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit fea40d6)
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7d9a4de2-402d-4657-8082-0388fe302134

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/overlay-guard-release-1.6

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size/M This PR changes 30-99 lines, ignoring generated files area/ci Issues or PRs related to CI workflows, GitHub Actions, automation kind/bug Categorizes issue or PR as related to a bug labels Jul 28, 2026
@dosubot dosubot Bot added the backport-previous Backport target — previous release line label Jul 28, 2026
@myasnikovdaniil

Copy link
Copy Markdown
Contributor Author

Closing in favour of #3471, which now solves this properly rather than by turning the overlay off.

This PR carried the interim fix: skip the current-main overlay for non-main base branches. That stopped main's binaries being installed against release-1.6's charts, but it left release-line PRs testing the line's last release — for any package a PR does not rebuild, the committed ref is the released digest, so a component changed by an earlier backport is exercised as its pre-backport binary until the next rc.

#3471 instead adds build-release.yaml, which publishes cozystack-packages:<line> for every maintained release-<major>.<minor> branch the way build-main.yaml does for main, and points the overlay at ${{ github.base_ref }}. Each base branch then overlays its own generation, which removes both the cross-generation mismatch and the staleness.

release-1.6 still needs the change on the branch itself — for pull_request events GitHub builds the workflow from the merge ref — so this will be backported from #3471 once that merges, and #3437 picks it up on its next E2E run with no rebase.

myasnikovdaniil added a commit that referenced this pull request Aug 3, 2026
… artifacts (#3471)

## What this PR does

The PR finalize job overlays image refs for every package a PR did not
rebuild, so e2e and the installer do not test last-release images for
everything outside the PR's build matrix. It could only ever read
`cozystack-packages:main`, which meant a **release-line PR was handed
main's binaries to run against its own line's charts**.

#3437 is the demonstration: a one-line change on
`release-1.6` deactivating an app failed install deterministically,
twice, with

```
helmrelease/backupstrategy-controller: Helm install failed …
error validating data: SchemaError(github.com/cozystack/cozystack/pkg/apis/core/v1alpha1.Option.spec):
unknown model in reference: "github.com~1cozystack~1cozystack~1pkg~1apis~1core~1v1alpha1.OptionSpec"
```

main's `cozystack-controller` served an aggregated OpenAPI that
`release-1.6`'s charts could not validate against. Nothing in that PR
was broken; the lane was. Both branches carry `option_types.go` and key
`OptionSpec` identically in the committed generated OpenAPI, so this is
a generation mismatch at runtime, not a codegen drift.

This PR fixes it by giving every base branch its own artifact to overlay
from, rather than by turning the overlay off.

### The change

1. **`pull-requests.yaml`** reads `cozystack-packages:${BASE_REF}`
(`github.base_ref`) instead of a hardcoded `:main`.
2. **`build-release.yaml`** (new) publishes that artifact for maintained
`release-<major>.<minor>` branches exactly as `build-main.yaml` does for
main: every image tagged with the branch, and the whole packages tree
pushed with each reference digest-pinned to what the run just built.

An earlier revision of this branch simply skipped the overlay for
non-main bases. That fixed the wrong-images problem but left
release-line PRs testing their line's *last release*: for any package
the PR did not rebuild, the committed ref is the released digest, so a
component changed by an earlier backport was still exercised as its
pre-backport binary until the next rc. Per-line artifacts remove that
gap too, which is why the guard was replaced rather than kept.

### Three deliberate choices

**The trigger matches line branches only** (`release-[0-9]+.[0-9]+`).
The per-release and rc staging branches `promote-rc.yaml` and
`tags.yaml` create — `release-1.6.1`, `release-1.6.0-rc.4` — must not
trigger a full rebuild; their images come from the tag build and
rebuilding them is waste.

**`WRITE_CACHE` stays `0`.** `CACHE_REGISTRY/<img>:buildcache` is a
single ref per image and `build-main.yaml` is deliberately its only,
serialized writer so concurrent builds cannot race on the cache manifest
— the 409 class #2711 fixed for image tags. A line build can overlap a
main build, so writing here would reintroduce that race. Line builds
read the cache.

**A missing artifact still degrades to committed refs, but says so.** On
a release line it emits a `::warning::` naming the branch. Silent
degradation is indistinguishable from a working overlay, which is how a
mis-specified branch filter could hide for a whole release cycle.

### Cost

One `make build` per push to a maintained line — in practice per merged
backport. That is the price of release-line PRs testing their line's tip
instead of its last release.

### Verification

`hack/overlay-main-images_test.bats` pins the artifact tag to the base
branch, rejects a hardcoded `:main` in either overlay step, and pins
`build-release.yaml`'s branch filter, image tag and `WRITE_CACHE: '0'`.
Mutation-checked: restoring `:main`, setting `WRITE_CACHE: '1'`, and
broadening the filter to `release-*` each fail a test. 13/13 green;
`actionlint` and `zizmor` clean.

Worth an explicit ack in review: the branch-filter pattern is the one
thing no local test can prove, since only GitHub evaluates it. If it
does not match, `build-release` never runs and the new `::warning::` is
what surfaces it on the next release-line PR.

### Backport

`release-1.6` needs this too — for `pull_request` events GitHub builds
the workflow from the merge ref, so a release-line PR only stops
receiving main's images once the change is on its base branch.
#3472 carried the interim guard and is closed in
favour of backporting this instead.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added a “Build release line” workflow that builds and publishes images
and the packages artifact for maintained `release-<major>.<minor>`
branches.

* **Bug Fixes**
* Updated PR workflow finalization to use base-branch–specific package
overlays, avoiding incorrect main-branch image references when targeting
release branches.
* Improved fallback behavior when the base-branch packages artifact is
unavailable.

* **Tests**
* Added workflow wiring tests to verify base-branch artifact usage and
that each maintained release line publishes its own packages artifact.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci Issues or PRs related to CI workflows, GitHub Actions, automation backport-previous Backport target — previous release line kind/bug Categorizes issue or PR as related to a bug size/M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant