fix(ci): restore e2e for fork PRs and close the merge-gate bypass - #3262

… upload reads

The fork build sets OCI_EXPORT_DIR to a relative `_out/oci`, but each image is
built with `make -C <pkg> image`, so buildx resolves the relative `dest=`
against the package directory — the archive landed in <pkg>/_out/oci/ while the
`if-no-files-found: error` upload reads repo-root `_out/oci/*.oci.tar`. Every
non-docs fork build would have failed at that upload (never caught, since
same-repo PRs leave OCI_EXPORT_DIR empty). Root it at github.workspace so the
dest resolves to repo-root _out/oci regardless of the sub-make cwd.

Reported by gemini-code-assist on #3262 (its abspath suggestion resolves against
the package dir under `make -C`, so this fixes it at the workflow level instead).

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

`count=$(ls _out/oci/*.oci.tar | wc -l)` runs under `shopt -s nullglob`, so
when nothing matches the glob expands to zero words and `ls` lists the current
directory instead — count is always >=1 and the "at least one archive" guard
never fires. A zero-archive build then falls through to `flux push` and defers
the real failure to an obscure image-pull error deep in the ~180-minute e2e
run. Glob into an array (empty under nullglob) and test its length; reuse it
for the push loop.

Reported by coderabbitai on #3262.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

…n-depth)

Belt-and-suspenders for the B2-class trap lexfrei flagged as latent in
image-tags. PUBLISH_* are 0 on fork PRs, so today the export path never emits a
versioned/floating tag — but if it ever did:

- image-tags: multiple --tag under `--output type=oci` write >1 manifest into
  the archive's index.json, and `skopeo copy oci-archive:<f>` then refuses it
  ("more than one image in oci"). Suppress the versioned/floating tags when
  OCI_EXPORT_DIR is set (macro-level $(if …), so `make -n` prints only the
  taken branch).
- packages/core/talos: gate its versioned/floating `skopeo copy docker://…`
  release copies on an empty OCI_EXPORT_DIR too.

The release (non-export) path is unchanged — versioned + :latest still emitted.
Covered by a new common-envs_test.bats case that forces PUBLISH_*=1 under
export.

Reported by gemini-code-assist on #3262.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

Address review feedback from IvanHunters on hack/common-envs_test.bats:70:
`grep -c` counts matching LINES, not occurrences. The image-tags macro
expands onto a single line, so every "exactly one tag survives under
OCI_EXPORT_DIR" assertion returned 1 no matter how many --tag flags were
actually emitted.

Proven vacuous by mutation: removing the OCI_EXPORT_DIR guard from
image-tags restores three --tag flags on that one line and the suite
stayed green. Switching to `grep -o … | wc -l` counts occurrences, and
the same mutation now fails the suite as it should.

Applied to the --output type=oci,dest= assertions too, which count
occurrences the same way and had the same blind spot.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

Address review feedback from IvanHunters on hack/common-envs_test.bats:
every existing case set OCI_EXPORT_DIR, so the OFF path — the one every
same-repo PR, main and release build still takes — had no regression
test at all. A refactor of the export wiring could stop pushing entirely,
or strip the release tags, and the suite would stay green.

Add a case asserting that with OCI_EXPORT_DIR unset buildx still pushes
(--push=1) and writes no archive, image-tags expands all three release
tags, and talos' skopeo copies still target docker:// with the
`[ -z "$(OCI_EXPORT_DIR)" ]` release gates inert.

Mutation-checked: making the PUSH/LOAD override unconditional (so export
mode leaks into the default path) fails this case.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

Address review feedback from coderabbitai on
.github/workflows/e2e-fork.yaml: the docs-only decision read only
`filename`, so a rename reported the NEW path alone. A PR renaming
packages/<app>/values.yaml -> docs/whatever.md therefore looked
docs-only, concluded "E2E Tests" green, and never ran the suite —
while in fact removing a code path. Both sides of a rename must be
under docs/ for a PR to count as docs-only.

`pulls.listFiles` is also capped at 3000 files, and a truncated list
could omit the only non-docs path. Fail closed at the cap instead of
ruling docs-only on a list known to be incomplete.

Feed both sides of a rename to TIA as well: the old path is a removal
the selector must see. Extra paths can only widen the selection, so
this cannot cause a suite to be skipped.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

Both comments stated that select-e2e.sh escalates to the full suite on
any unrecognised path, so fork TIA "can only narrow". That is not what
it does: it escalates for an unrecognised path INSIDE packages/, but a
path it does not recognise at all (hack/*.mk, for one) selects nothing,
the Chainsaw step is skipped, and the required status still concludes
success.

Found by coderabbitai in review. The behaviour is pre-existing and
identical for same-repo PRs through pull-requests.yaml, so it is tracked
separately in #3392 rather than fixed on the fork lane alone — which
would leave forks stricter than same-repo for the same diff. Correct the
comments here so the guarantee is not overstated in the meantime.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>

…3511)

## What this PR does

Closes #3392. Stacked on top of #3262 — that PR turns the `E2E Tests`
merge gate into a commit status, which is what makes this fail-open path
matter: a skipped required *job* used to pass branch protection either
way, but a status posted `success` with no suite run is an explicit
green verdict on untested code.

`hack/select-e2e.sh` silently selected nothing for a changed path it did
not recognise outside `packages/`. Both lanes read an empty selection as
"skip Chainsaw" and then report green. The worked example from the issue
is `hack/common-envs.mk`, which supplies the tag, push and output flags
of every image in the tree and matched nothing: `full_suite_pattern`
covered `hack/*.sh` and `hack/*.bats` but not `hack/*.mk`. Walking the
tree turned up five more in the same position — `pkg/`, `tools/`,
`go.mod`/`go.sum`, `hack/lib/` and `hack/buildkitd.toml` — plus
`.github/workflows/e2e-fork.yaml`, which #3262 adds while
`pull-requests.yaml` was already escalated.

As the issue argues, a blanket "empty selection means full suite"
fallback would be the wrong fix, because it escalates exactly the inert
paths the design deliberately skips. Instead every changed path now
lands in one of three classes by an explicit rule:

- `full_suite_pattern` — shared build inputs and the e2e harness:
`packages/library/`, `packages/core/`, the Go trees (`api/`, `cmd/`,
`internal/`, `pkg/`), the codegen under `tools/`, `go.mod`/`go.sum`,
`hack/*.sh|*.bats|*.mk`, `hack/lib/`, `hack/buildkitd.toml`,
`hack/e2e-*.yaml`, the `Makefile`, and the workflows that run the suite.
- `inert_config_pattern` — paths that cannot affect what e2e exercises:
`examples/`, `.github/` (bar the e2e workflows), `.claude/`, `.gemini/`,
`img/`, `hack/testdata/`, the codegen boilerplate header, and the
top-level meta files (`LICENSE`, `.gitignore`,
`.pre-commit-config.yaml`, `.coderabbit.yaml`).
- the PackageSource graph for `packages/(apps|system|extra)/`,
unchanged.

A path matching none of them escalates to the full suite and logs which
path forced it, so the next path someone adds fails safe rather than
fails open. The two lists are checked in that order, so a specific
escalation beats a broad inert directory — `.github/` is inert,
`.github/workflows/e2e-fork.yaml` is not. `*.md`, `docs/` and
`dashboards/` stay ahead of both, so a `README.md` under
`packages/core/` does not inherit that tree's escalation.

The second commit removes the `|| [ -z "$SELECTED_APPS" ]` arm from the
Chainsaw step in both lanes. Under a step gated on `skip != 'true'` it
could never decide anything the `FULL_E2E` test had not, while the
comment above it advertised "full suite when the selection is empty" in
the file whose whole point is that a green status means tests ran.

### Deliberate choices worth a second opinion

- **Which workflows escalate.** The four that actually invoke the suite
(`pull-requests`, `e2e-fork`, `e2e-tag`, `nightly`), enumerated rather
than matched by prefix so an unrelated workflow does not burn a full
run. That does mean an `e2e-tag.yaml` or `nightly.yaml` edit now costs a
full PR suite even though the lane it changes is not the PR lane. Easy
to narrow to the two PR lanes if that is the wrong trade.
- **`go.mod`/`go.sum` escalate.** A dependency bump changes the shipped
binaries, so this is the honest classification, but it does make
Renovate's Go PRs pay for a full suite.

### Verification

- `hack/select-e2e_test.bats`: 25 tests green (15 existing, 10 new),
covering both sides of the classification.
- Mutation-checked rather than asserted: reverting the fall-through
fails the unclassified-path test; dropping `.github/` from the inert
list fails the non-e2e-workflow test; swapping the order of the two list
checks fails the existing `pull-requests.yaml` test.
- The per-entry escalations are belt-and-braces by design: removing
`hack/*.mk` from the pattern still escalates via the fall-through, so
that test pins the outcome rather than the pattern entry. Called out so
nobody reads it as stronger coverage than it is.
- `make bats-unit-tests`: 417 tests, 0 failures. `actionlint` clean on
both workflows.
- Self-check: this branch's own diff selects the full suite (it touches
both e2e workflows and a `hack/*.sh`).

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the diff
file by file. The only `hack/`-related trigger is `cozystack/ccp`, which
gates on `hack/package.mk` and `hack/common-envs.mk` existing as anchor
files and on what `make generate` does — nothing here is moved or
renamed, neither anchor file is touched, and no make target changes
behaviour (`BATS_UNIT_FILES` already globbed `hack/*.bats`). The rest of
the diff is e2e workflows and agent docs, which appear nowhere in the
map.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(ci): Test-Impact Analysis now classifies every changed path explicitly and escalates an unrecognised one to the full E2E suite. Previously a path the selector did not recognise (`hack/*.mk`, `pkg/`, `go.mod`, `hack/lib/`, the fork E2E workflow) selected no suites, which both E2E lanes read as "nothing to test" and reported the required `E2E Tests` gate green with no suite run.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved end-to-end test selection so unrecognized or high-impact
changes trigger the full suite instead of being skipped.
* Inert-only changes now safely skip testing, while empty selections no
longer trigger the full suite.
* Full-suite testing is triggered explicitly by the `full-e2e` label or
applicable change types.

* **Tests**
* Added coverage for build files, source code, workflows, scripts,
documentation, metadata, and unclassified paths.

* **Documentation**
* Clarified end-to-end test selection, escalation rules, skipped file
categories, and workflow handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

…#3569)

## What this PR does

A run joins its concurrency group before any job-level `if` is
evaluated, so `prepare`'s guard in `backport.yaml` is too late to help.
By the time it works out that a label event is irrelevant, that run has
already cancelled whatever was in flight for the same PR. Labels set
with the default `GITHUB_TOKEN` start no run, which is why the in-repo
labelers never showed this. Third-party GitHub Apps do start runs, and
they arrive in bursts.

The delivery survived, which is why this went unnoticed for so long. The
old guard admitted a run on the PR's cumulative label set, so the last
label of a burst, the one nothing cancels, requalified and redid the
work. In the bursts on record the kill lands within seconds of the merge
run starting, well before the cherry-pick, so the cost was a restart
rather than a lost backport, recorded as nothing more than a cancelled
run sitting next to a green one.

Two changes to the concurrency key, and both are needed. A label event
that requests no backport now goes into a group of its own, so it can
neither cancel nor displace the run doing the work. A label event that
does request one queues behind that run instead of killing it, because
`backport` and `backport-previous` are separate requests and both have
to finish. Splitting alone would let `backport` cancel the
merge-triggered run. Queuing alone would let an unrelated label evict a
pending request, since the group holds a single pending run and a newer
one replaces it.

`prepare`'s condition is scoped in the same commit. It read the PR's
cumulative label set, so once a merged PR carried `backport`, any later
unrelated label re-entered the job to redo a backport already delivered.
That run used to get cancelled by the next label in the burst. After the
change above it runs to completion, so the scoping has to land here
rather than as a follow-up.

The coupling runs the other way too, and it is the sharper half. That
redundant run is exactly what redelivered the backport the burst had
just killed, so narrowing the guard on its own would convert a noisy
delivery into a missing one. Neither change is safe to land without the
other.

Both jobs also gain a `timeout-minutes` ceiling, which is a consequence
of the queuing rather than housekeeping. A cancelling key disposed of a
stuck run by killing it; queuing makes the next genuine request wait
behind it instead, on the six-hour job default, which turns one wedged
job into a six-hour hole in the release line. That ceiling reaches a
stuck job and nothing else: a run can also wedge at the run level with
every job already finished, and `timeout-minutes` has no run-level
equivalent, so nothing in the workflow reaches that state.
`docs/release.md` now tells an operator to look for a run still in
flight and clear it before re-applying the label, because the retry
queues behind it rather than replacing it. The ceiling bounds execution,
not the wait for a runner, and thirty minutes is generous rather than
calculated. Execution on run 29844315422 was six seconds for `prepare`
and two and sixteen for the matrix legs, against 9m24s, 5m28s and 5m45s
of runner queue for the same three jobs. That queue is a third unbounded
case: GitHub's limits page says "job execution time" for the six-hour
job cap while spelling out that the 35-day run cap "includes execution
duration, and time spent on waiting and approval", and
[orgs/community#50926](https://github.com/orgs/community/discussions/50926)
shows a job sitting six hours on "Waiting for a runner to pick up this
job" under `timeout-minutes: 5`. So the value is picked to leave room
for a cherry-pick far larger than any on record while replacing the
six-hour default, and nothing here is sized against the queue, because
nothing here can be.

This removes a mitigation, and the trade is worth naming on both sides.
Under an unconditional cancel a run wedged at the run level cleared
itself, because the next event killed whatever was sitting in the group.
That is the same behaviour that killed live backports, so the mitigation
is inseparable from the bug being fixed: keeping it means keeping label
events that cancel a backport in flight. What it buys is a change of
failure mode rather than one fewer failure. A backport lost silently,
discovered weeks later as a fix missing from a release line, becomes a
delay the operator can see in the run list, with the recovery written
into `docs/release.md`. There is no lossless alternative here: `queue:
max` removes the eviction but only by dropping the conditional cancel
entirely, and the group key cannot tell a manual retry from a second
genuine request.

The run cited above, 31232645133, is still live and retained on purpose:
it is the only observed instance of the run-level wedge, so clearing it
would remove the evidence for the paragraph that describes it. Its
exposure is one pull request, #3262, which carries no backport label, so
nothing is queued behind it.

The old condition also carried a disjunct that could never decide
anything. A `labeled` payload already lists the new label in
`pull_request.labels`, so `contains` was true whenever the explicit
`github.event.label.name` check was, and the name check only ever agreed
with a `contains` that had already matched. The restructure drops it by
construction rather than by deletion: the name check is now the whole of
what a `labeled` event may match on.

### Why this no longer touches `pull-requests.yaml`

`main` fixed it there by splitting the group key, then refined that so a
`full-e2e` label stays in the main group instead of publishing a second
`E2E Tests` status on one SHA. That is the better mechanism, it neither
queues nor delays, and `gate-concurrency-contract.bats` now pins
`cancel-in-progress: true` in that file, so what this PR originally did
would turn it red. Nothing was left to add there, so the hunk is dropped
and its pin with it. The split-key idea is applied in `backport.yaml`
instead, where it did not exist.

One claim from the earlier description is withdrawn. It said the
leftover risk was a duplicate backport PR, opened when the original had
merged and its branch had been deleted before the queued run pushed.
`main` has since gained a guard that lists closed PRs on the
`backport-<n>-to-<target>` branch and skips the target when one of them
merged. Branch deletion does not defeat it: `GET
/branches/backport-3510-to-release-1.6` is 404 while the same head
filter on `/pulls?state=closed` still returns the merged PR. So a
redundant run only cost runner minutes, and the scoping above removes
the run.

### Screenshots

Not applicable, no UI change.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` against the
diff. The only workflow it cites is `tags.yaml`, for the release-prep
behaviour `cozystack/ccp` documents. Every `hack/` trigger it lists
either names a specific file another repository mirrors or gates on, or
covers moving and renaming under `hack/`. This PR appends tests to a
contract file the map does not name, and moves nothing. No package,
chart, values file, CRD, image reference, namespace, variant or node
prerequisite in the diff.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(ci): stop `labeled` pull request events from cancelling the automatic backport run already in flight. A burst of labels from a third-party app could previously kill the backport the merge had just started, leaving the work to be redone by whichever label arrived last.
```