feat(release): add backport-audit, a pre-release backport triage tool - #3527

What this PR does

Adds cmd/backport-audit, which answers one question before a release or an rc is cut: did everything labelled for backport actually land on the release branch? It prints the URLs of the ones that did not and exits non-zero, so it can gate the cut. docs/release.md is routed through it.

The gap it closes is in the docs it replaces. docs/release.md's "Cherry-pick triage before a patch" told the release manager to inventory backports with gh pr list --search "is:merged label:backport", annotated "merged but not yet on release-X.Y" — which that command cannot tell you. It lists every PR ever labelled, whether or not the change reached the branch, and whether it arrived is the only thing that matters at that point in a cut. Answering it by hand means reading the branch history per PR, so backports go missing quietly.

A backport label does not name a branch: backport.yaml resolves the target from getLatestRelease at merge time, so the same label means release-1.5 on a PR merged in June and release-1.6 on one merged in August. The audit reproduces that rule rather than trusting the label text, then establishes landing from three independent kinds of evidence — the merge commit already being reachable from the branch, a linked backport PR (bot head branch or a Backport of #N body reference), or the branch's own history (the bot's merge subject, an identical commit subject, an -x cherry-pick reference). Each candidate comes out as in-branch, backported, pending, dropped (with whatever reason someone recorded on the closed backport PR) or MISSING.

$ go run ./cmd/backport-audit release-1.5

=== release-1.5 === 33 candidate PRs: MISSING=4 pending=9 dropped=3 backported=17

  MISSING -- labelled for this line, no trace of it here (4):
    https://github.com/cozystack/cozystack/pull/2936
      #2936 fix(kubernetes): make the default md0 node group removable
      label=backport author=myasnikovdaniil merged=2026-06-30 -- no backport PR, nothing on branch

What running it found

Against release-1.6, release-1.5 and release-1.4, over the 271 merged PRs that carry a backport label:

Every one of the 9 MISSING items merged before 2026-07-03 07:53Z, which is when #3155 landed. Before it, conflict_resolution was passed as a top-level input instead of nested under experimental, so the action ignored it and fell back to fail — which on a conflicting cherry-pick opens no PR and reports no failure. Those backports were dropped with no draft to find and no red check to notice. The correlation holds in both directions: every candidate merged after #3155, across all three lines, has a backport PR. So this is a bounded debt pile from that window rather than an ongoing leak, and docs/release.md now records the signature so a future cluster of MISSING verdicts is read as bot failure rather than as maintainers forgetting.

All 9 MISSING verdicts were confirmed absent independently of the tool, at diff level: every file each PR touched still differs from main on the target branch, and for 7 of 9 at least one touched file does not exist on the branch at all. Two were checked by hand — release-1.4 still carries the pre-#2704 $host := .Values._namespace.host | default ... form in packages/extra/info/templates/kubeconfig.yaml, and data/versions appears nowhere in its tags.yaml. No false positives in that run.

Verification

The implementation was cross-checked against an independently written prototype rather than against its own re-reading of the rules: run over the same three lines, the JSON is byte-identical across all 68 verdicts, as is the 104-line text report and the exit code.

That parity also covers one deliberate shortcut. Presence on a branch could be one git merge-base --is-ancestor per candidate per branch, 813 subprocesses for three lines; instead the history walk the audit already makes for subjects and cherry-pick references also collects the reachable commit oids, and a set lookup answers the same question. Reachable from the branch tip is exactly what --is-ancestor tests, and identical output over 68 verdicts is the evidence. It is not faster end to end — roughly ten seconds either way, because wall-clock is dominated by the gh API calls, not by local git.

gofmt and go vet are clean. Color follows cmd/check-readiness: auto-disabled on a non-TTY and under --json, with --no-color to force it off, so piping into jq stays clean.

Limits

Deliberately stated in the README rather than papered over. A MISSING verdict is a prompt to check, not proof of absence: a hand-backport that was squash-merged, reworded and referenced no original PR is invisible to every evidence layer and reads as MISSING. A backport label added long after merge, once the release line has moved on, resolves to the newer line. And the audit reports that something is missing, never why.

make generate does not apply: this adds a standalone main package with no API types, no CRDs and no package values.yaml, so hack/update-codegen.sh has nothing to regenerate from it.

Screenshots

Not applicable — no UI change. Sample CLI output is above.

Downstream repositories

Walked the trigger map in docs/agents/contributing.md file by file against this diff, which is: cmd/backport-audit/main.go and cmd/backport-audit/README.md (both new), docs/release.md, docs/agents/releasing.md, and one .gitignore line for the root binary the cmd/ convention requires.

Nothing in it reaches a downstream repository. No package under packages/apps/ or packages/extra/ is added, renamed or removed, and no values.schema.json, values.yaml default or version enum changes, so the website generator lists and the hand-written Terraform provider schemas are untouched. No ApplicationDefinition semantics, release.prefix, output Secret or Service name, namespace, platform variant or bundle changes. Nothing under hack/ moves or is renamed and no make target changes behaviour, so ccp's anchor files and external-apps-example's vendored copies of hack/package.mk and hack/update-crd.sh are unaffected. No node prerequisite in hack/e2e-prepare-cluster.bats changes, so talm's charts/cozystack/ and the Ansible prepare playbooks do not need a matching PR. No telemetry metric or label, no cozyhr annotation, no cozy-proxy label or annotation contract. The website's "developer tooling" trigger covers cozyvalues-gen, cozypkg and the package Makefiles, which contributors use on packages; this is maintainer release tooling whose process doc lives in this repository and is not mirrored to the site.

• No downstream repository is affected by this change
• cozystack/website - follow-up:
• cozystack/terraform-provider-cozystack - follow-up:
• cozystack/ansible-cozystack - follow-up:
• cozystack/ccp - follow-up:
• cozystack/talm - follow-up:
• cozystack/cozyhr - follow-up:
• cozystack/cozy-proxy - follow-up:
• cozystack/cozystack-telemetry-server - follow-up:
• cozystack/external-apps-example - follow-up:
• cozystack/examples - follow-up:

Release note

feat(release): add `backport-audit`, a maintainer tool that reports which `backport` / `backport-previous` PRs have not landed on a release branch — as missing outright, waiting on an open backport PR, or deliberately dropped — and exits non-zero while anything is outstanding, so a patch cut can gate on it