Skip to content

feat(ci): auto-apply triage labels to issues - #3574

Merged
Aleksei Sviridkin (lexfrei) merged 1 commit into
mainfrom
feat/issue-triage-labeler
Aug 12, 2026
Merged

feat(ci): auto-apply triage labels to issues#3574
Aleksei Sviridkin (lexfrei) merged 1 commit into
mainfrom
feat/issue-triage-labeler

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Issues that arrive through the API or gh issue create bypass the issue templates, so they carry no triage/* label and nothing in the repository adds one afterwards. Listing every open issue through the same endpoint the workflow uses, gh api --paginate "/repos/cozystack/cozystack/issues?state=open&per_page=100", returns 567 entries, 171 of them pull requests, at the time of writing. That leaves 396 open issues, and 252 of them carry no triage label at all. Every count in this description is a point-in-time reading of a backlog that keeps moving, so a later run of the same query will differ by a few. The triage queue is impossible to filter on in that state, and there is no way to see how much of the backlog nobody has looked at.

This adds a workflow that labels an issue when it is opened or reopened, plus a daily sweep that backstops labels removed by hand and works through the existing backlog. An issue somebody has assigned, or marked with one of priority/critical-urgent, priority/important-soon, priority/important-longterm or epic, gets triage/accepted, since a maintainer has already looked at it. Everything else gets triage/needs-triage. An issue already carrying any triage/* label is left alone, so the sweep never overwrites a decision someone made by hand, and pull requests returned by the same listing endpoint are skipped. Against the listing above, the sweep applies triage/accepted to 39 issues and triage/needs-triage to 213, and touches none of the 144 that are already triaged.

Deleting a triage label by hand does not undo any of this. An issue left carrying no triage/* label is relabeled by the next sweep, so a triage decision is changed by replacing the label. AGENTS.md's labeling section now says so, because that is the file somebody reads before labeling by hand.

Two conditions decide what counts as a signal. Membership of stale.yaml's exempt-issue-labels is necessary: triage/accepted is itself on that list, so applying it does more than reset a clock, the issue stops being auto-closable, and a signal that is not already exempt would hand out that permanent reprieve as a side effect of being labeled. priority/backlog, the one tier stale.yaml deliberately leaves reapable, is excluded for exactly that reason.

Membership is not sufficient, which is why the signal list is not simply the exempt list. lifecycle/frozen is exempt and still not a signal: a lifecycle/* label states what the stale bot may do with an issue and says nothing about whether anybody reviewed it. Freezing an untriaged issue to keep the bot off it is the case where reading it as accepted is wrong. Of the five open issues whose only signal would have been lifecycle/frozen, one is blocked on an upstream project and carries help wanted, and one is an older feature request that an open epic now covers. Neither is ready to be worked on, and stamping either accepted would have kept it out of triage for good, since the sweep never revisits an issue that already carries a triage label. epic passes the same test from the other side: it is a maintainer-authored planning artifact rather than an inbound report, and all eight open issues whose only signal is epic are roadmap trackers that needs-triage would describe falsely. A contract test pins the rule and not the instance, so no label from the lifecycle/* namespace can quietly become a signal later. security/* stays out on the same grounds, though the case is closer: those labels record what is known about a vulnerability rather than whether anyone triaged the report, and the cost of being wrong runs the cheap way, since a needs-triage on a confirmed report is one click to remove while a wrong accepted is permanent because the sweep never revisits.

One caveat stated plainly: the signals stand for a maintainer decision already recorded on the issue, which is a weaker claim than the one triage/accepted makes in .github/labels.yml, and priority/important-longterm says outright that the work may not be staffed. Of the two labels this workflow can write it is still the closer one, since needs-triage on an issue somebody already prioritised or took asks for a triage that has happened, and a maintainer who disagrees replaces the label in one click. An assignee is the single signal that is neither a label nor already exempt, so an issue whose only signal is an assignee does gain a permanent reprieve. That is the intended reading rather than a side effect: somebody took ownership, which makes it a known long-tail task rather than an abandoned one, and it is the same argument stale.yaml's own comment makes for exempting triage/accepted in the first place.

Three operational details. First, the sweep paces its writes a second apart and caps a run at 400, because GitHub allows 80 content-generating requests a minute and 500 an hour, this client carries no throttling plugin and no retries, and a throttled write answers 403, which is exempt from retry anyway. Unpaced, the first pass over the backlog would not slow down when throttled, it would fail the job partway and leave the backlog half labeled. Whatever a run leaves behind stays unlabeled, so the next one picks it up. Second, adding a label bumps an issue's updated_at, which stale.yaml reads as activity, so the first sweep restarts the 60-day staleness clock on every issue it writes to, all 252 of them, and un-marks the 32 that carry lifecycle/stale today. Both are deliberate and one-time: the backlog gets triaged now and ages again from there, rather than being closed unread while it is still unlabeled. Third, a run writes a job summary with the counts, the issue numbers under each label, and both remainders it can leave behind: the write cap's deferral, and how much of the listing it never reached when a refusal stopped it early. A sweep over the backlog logs a line per issue, and the summary is what makes the outcome readable without paging through the 252 the current backlog produces.

A write that fails does not take the rest of the run with it, but the two failure shapes are told apart rather than counted together. Any status other than a 403 or 429 lets the run carry on, most often a 404 or 410 from an issue transferred or deleted mid-sweep, since a failure about one issue says nothing about the next. A 403 or 429 means the token lost issues: write or the pacing stopped being enough, which applies to every write left in the run, so the sweep stops at the first one and the job fails loudly. Repeating a rejected request up to the cap is what gets an integration throttled harder.

workflow_dispatch takes a dry_run input that logs what would be labeled without writing anything. It is on by default, so a manual run writes nothing until somebody unchecks the box, and schedule is the one event that writes without being asked. The cap applies in dry-run too, so a dry run predicts what a real one would do instead of printing a longer list nobody gets. The job asks for issues: write and everything else is read-only, the action it uses is pinned by digest, and a 20-minute timeout keeps a hung call from holding the queue for the default six hours.

No pull request lane can exercise this workflow, so its first real execution is a sweep over every open issue with nothing having run it before. hack/issue-triage-contract.bats pins the executable lines that bound that blast radius: the pull-request skip, the already-triaged skip, the pacing rate and the write cap, the job timeout, the failure handling, the digest-pinned action, the token scopes, the serialised queue, the dry-run default and the derivation that does not depend on it, the cron ordering against stale.yaml, that both labels it writes exist in .github/labels.yml, that no accepted-signal label sits outside stale.yaml's exempt list or inside the lifecycle/* namespace, and that the summary is written with the await that makes it flush and accounts for every outcome a run can end on. Each assertion was checked against a mutated copy of the workflow to confirm it goes red rather than merely existing, with the mutations run through hack/cozytest.sh rather than bats. The two runners disagree about negated assertions, so a pin that bats reports as live can be dead under the runner CI actually uses, and every negative assertion in the file captures grep's status instead of leading with !.

Suggested first step after merge: run the workflow by hand and read the summary before the 05:53 cron arms itself. The box arrives checked, so that run costs nothing.

Screenshots

Not a UI change, so there is nothing to show.

Downstream repositories

I walked the trigger map in docs/agents/contributing.md against the diff. The diff adds two files, .github/workflows/issue-triage.yaml and hack/issue-triage-contract.bats, and appends one line to AGENTS.md. No row matches: it adds and renames no package, changes no chart values, no schema, no version enum, no ApplicationDefinition, no CRD, no namespace, no variant or bundle, no label or annotation any downstream repository matches on, no metric and no release asset. Its hack/ rows name hack/upload-assets.sh, hack/e2e-prepare-cluster.bats, hack/package.mk, hack/update-crd.sh and hack/common-envs.mk, which downstream repositories copy, anchor on or mirror by hand, plus one row for moving or renaming anything under hack/; a new bats file added next to them is none of those. The map has no row for the agent documentation, and no downstream repository copies AGENTS.md.

Release note

feat(ci): new and reopened issues now get a `triage/*` label automatically, and a daily sweep labels any open issue that has none: `triage/accepted` when it is assigned or carries an `epic` or a priority above `priority/backlog`, `triage/needs-triage` otherwise.

Summary by CodeRabbit

  • New Features

    • Added automated issue triage triggered by new or reopened issues, scheduled reviews, or manual runs.
    • Issues can be labeled as needing triage or accepted, with pull requests and already-triaged items excluded.
    • Added dry-run support and reporting for triage results.
  • Documentation

    • Added guidance for replacing existing triage labels when making decisions.
  • Tests

    • Added coverage for workflow behavior, permissions, scheduling, failure handling, and safety limits.

@lexfrei
Aleksei Sviridkin (lexfrei) marked this pull request as ready for review August 6, 2026 07:50
@github-actions github-actions Bot added size/XL This PR changes 500-999 lines, ignoring generated files area/ci Issues or PRs related to CI workflows, GitHub Actions, automation kind/feature Categorizes issue or PR as related to a new feature labels Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds an issue-triage workflow and a Bats contract suite. The workflow classifies issues, applies triage labels, supports scheduled and manual sweeps, limits writes, handles refusals, and reports results.

Changes

Issue triage automation

Layer / File(s) Summary
Workflow entry and classification
.github/workflows/issue-triage.yaml, hack/issue-triage-contract.bats, AGENTS.md
The workflow defines triggers, permissions, concurrency, labels, accepted signals, and issue classification. Guidance describes replacement of existing triage/* labels. Contract tests verify skip rules, action digests, permissions, concurrency, stale compatibility, schedule ordering, and label declarations.
Issue event label handling
.github/workflows/issue-triage.yaml, hack/issue-triage-contract.bats
Issue events apply triage labels or log dry-run results. Contract tests verify manual dry-run defaults and runtime input handling.
Sweep execution and write handling
.github/workflows/issue-triage.yaml, hack/issue-triage-contract.bats
Scheduled and manual sweeps paginate open issues, enforce pacing and write caps, isolate per-issue failures, stop on HTTP 403 or 429 responses, report totals, and fail on refused writes. Contract tests verify these controls.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant IssueEvent
  participant IssueTriageWorkflow
  participant GitHubIssuesAPI
  IssueEvent->>IssueTriageWorkflow: trigger issue, schedule, or dispatch
  IssueTriageWorkflow->>GitHubIssuesAPI: retrieve open issues
  GitHubIssuesAPI-->>IssueTriageWorkflow: return paginated issues
  IssueTriageWorkflow->>IssueTriageWorkflow: classify issue and enforce write limits
  IssueTriageWorkflow->>GitHubIssuesAPI: apply triage label
  GitHubIssuesAPI-->>IssueTriageWorkflow: return write result or refusal
Loading

Suggested reviewers: kvaps, myasnikovdaniil

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: automatically applying triage labels to issues through CI.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/issue-triage-labeler

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/issue-triage.yaml:
- Around line 142-146: Update .github/workflows/issue-triage.yaml lines 142-146
to detect multiple triage labels and reconcile them to the documented
authoritative state instead of skipping every triage-labeled issue. Before the
event-driven addLabels call at lines 163-167, re-read and reclassify the issue;
likewise, before each sweep-item write at lines 203-220, re-read and reclassify
the issue so concurrent label changes cannot produce contradictory labels.
- Around line 189-194: Update the issue-processing flow around github.paginate
to use github.paginate.iterator with for-await page iteration, processing issues
incrementally and breaking once writes reaches MAX_WRITES_PER_RUN. Revise
deferred-count handling so it does not depend on fetching or retaining remaining
pages after the write cap is reached.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9c22e6e7-187f-47bc-ae6d-63c68acd84e8

📥 Commits

Reviewing files that changed from the base of the PR and between b4e2031 and 11d63da.

📒 Files selected for processing (2)
  • .github/workflows/issue-triage.yaml
  • hack/issue-triage-contract.bats
🚧 Files skipped from review as they are similar to previous changes (1)
  • hack/issue-triage-contract.bats

Comment thread .github/workflows/issue-triage.yaml
Comment thread .github/workflows/issue-triage.yaml
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Aleksei Sviridkin (lexfrei) added a commit that referenced this pull request Aug 7, 2026
## What this PR does

The comment above the top-level `permissions:` block said jobs request
the minimum extra scopes on top of it. That's backwards. A job-level
block replaces the workflow-level one, and [every scope it does not name
is set to
`none`](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#permissions).
Write `packages: write` alone in a job and it loses `contents: read` and
can't check the repo out.

Same wording in every file that carried the old line. It matches what
#3574 uses for the workflow it adds.

`backport.yaml` and `pull-requests.yaml` also appear in open #3569. Its
changed lines there are the `cancel-in-progress` expressions two lines
below the comment, and a three-way merge is clean in either order.

Comment-only: every changed line starts with `#`, the files still parse,
and actionlint reports the same 45 pre-existing findings before and
after.

### Screenshots

Not a UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

Walked the trigger map against the diff. The only row that names a file
here is the ccp one on `.github/workflows/tags.yaml`, and it triggers on
release-prep behaviour, which this doesn't touch.

### Release note

```release-note
docs(ci): the comment above the top-level `permissions` block in the workflow files now says a job-level block replaces it instead of adding to it
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Clarified workflow permission guidance across automation processes.
  * Documented that the default token permissions are read-only.
* Clarified that job-level permissions replace top-level defaults and
must explicitly declare all required scopes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@lexfrei
Aleksei Sviridkin (lexfrei) force-pushed the feat/issue-triage-labeler branch 2 times, most recently from aee5c6c to 5047409 Compare August 9, 2026 22:15

@IvanHunters IvanHunters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary: LGTM (one non-blocking note)

Reviewed the full diff (workflow + bats contract) and the repo state it depends on. This is a clean, tightly scoped, and unusually well-documented change.

Verified independently against the checkout:

  • Both applied labels (triage/needs-triage, triage/accepted) and all five accepted-signal labels exist in .github/labels.yml.
  • All five accepted-signal labels are a subset of stale.yaml's exempt-issue-labels, so triage/accepted never widens the stale exemption set (the invariant the workflow claims for itself).
  • stale.yaml cron (04:37) runs before the sweep (05:53), as the contract requires.
  • actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b matches the v7 tag digest, so the supply-chain pin is correct.
  • All 14 assertions in hack/issue-triage-contract.bats pass locally, and make unit-tests (run by pull-requests.yaml) auto-discovers hack/*.bats, so the contract actually executes in CI rather than being decorative.
  • Rate-limit math is correct and conservative: 1 write/s is 60/min (< 80/min secondary limit) and the cap of 400 stays under 500/h.
  • The dryRun derivation is fail-safe across every event x input combination: schedule writes, workflow_dispatch defaults to dry-run, and any future trigger added to on: starts out dry.
  • Permissions are minimal (contents: read plus a job-scoped issues: write), and there is no ${{ }} interpolation into the script: block.

Non-blocking note:

  • issue-triage.yaml, event path (if (context.eventName === 'issues')): the single apply() call on the issues: opened/reopened trigger is not wrapped in try/catch, unlike the sweep path which discriminates 403/429 from other statuses explicitly. A transient API failure on this (most frequent) trigger fails the job red instead of degrading. It is self-healing, since the next daily sweep backstops the unlabeled issue, so this is a consistency observation rather than a bug. Given that every other decision in this file carries an explicit rationale, it would be worth either handling it the same way or noting why a red run is the intended signal here.

Nothing blocking.

IvanHunters
IvanHunters previously approved these changes Aug 10, 2026

@IvanHunters IvanHunters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This adds .github/workflows/issue-triage.yaml (event-driven labeling on opened/reopened plus a daily sweep) and hack/issue-triage-contract.bats (a structural contract suite pinning the workflow's safety invariants). I reviewed this statically: the diff, the full PR discussion, and cross-referenced stale.yaml, .github/labels.yml, and the issue templates on main.

Verdict: LGTM.

Security

Checked this specifically as a issues-triggered workflow:

  • No pull_request_target, no checkout of untrusted code.
  • The github-script block never interpolates issue.title or issue.body into a shell command — in fact it never reads them at all. Classification uses only issue.pull_request, issue.labels[].name, and issue.assignees.length, consumed as parsed JS objects via context.payload.issue, not string-interpolated into ${{ }} anywhere in the YAML. No script-injection surface.
  • Permissions are minimal: top-level contents: read, job-level issues: write only (job-level permissions: replaces the workflow default rather than extending it, per GitHub Actions semantics, and matches the identical pattern already used in stale.yaml/pr-labeler.yaml).
  • The action is pinned by commit digest (actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7) — verified against the live v7 tag on actions/github-script, it matches exactly.

No security concerns.

Design/necessity

The stated problem (issues created via API/gh issue create bypass the issue-template front matter that sets triage/needs-triage, so they carry no triage label) checks out against bug_report.md on main. The accepted-signal label list is a proper subset of stale.yaml's exempt-issue-labels, so granting triage/accepted doesn't quietly widen the stale-exemption surface — verified by diffing both files directly, and it's also the one thing the added bats suite cross-checks mechanically. Cron ordering (05:53 after stale's 04:37) is correct and enforced by a test that computes it rather than hardcoding it.

The write-pacing (1/sec, 400/run cap) and fail-fast-on-403/429 design is proportionate to the actual risk here: there is no PR lane that can exercise this workflow, so its first real run is a blind sweep over the entire open-issue backlog. Given that blast radius, the extra ceremony (contract tests) is justified rather than gold-plated, and it follows an existing precedent in this repo (promote-gate-contract.bats, release-freeze-contract.bats).

Prior review threads

Both CodeRabbit findings were properly worked through, not ignored:

  • Concurrent-write race (a label added by a human mid-sweep could end up alongside the bot's label): maintainer response quantifies the cost (one stray label, removed by hand) against the fix (doubling the request budget by re-reading every issue before every write). CodeRabbit withdrew the finding.
  • Non-incremental pagination: maintainer response notes the backlog is ~280 issues (3 pages), dwarfed by the 400-write cap, and that switching to paginate.iterator would turn the "deferred" count into a lower bound rather than an exact figure. CodeRabbit withdrew this one too, and logged it as a learning to revisit if the backlog grows.

Both resolutions are sound engineering trade-offs backed by real numbers, not hand-waving.

Non-blocking notes

  1. The event-driven path (context.eventName === 'issues') calls apply(issue, label, false) directly and isn't wrapped in the same try/catch that shields the sweep from a single 403/429 — an isolated rate-limit hit on a single opened event will fail that job run outright. Low impact (the daily sweep will pick up the unlabeled issue regardless), but worth a one-line acknowledgment if this surprises anyone in the Actions history.
  2. hack/issue-triage-contract.bats is explicitly a structural/grep-based contract, not a behavioral test (the file says so itself). That's a reasonable, disclosed trade-off given there's no JS runtime guaranteed in the checks job, and it matches existing repo precedent — just flagging for anyone expecting unit-test-style coverage of the classification logic.

Nice level of self-documentation in the workflow comments — the rationale for every constant (rate limits, cap, cron offset) is traceable to a real GitHub constraint rather than an arbitrary number. Good to merge.

@myasnikovdaniil

Copy link
Copy Markdown
Contributor

Aleksei Sviridkin (@lexfrei) I checked permissions, pinning and the injection surface, all fine - issues: write only on the job, no ${{ }} anywhere in the file, digest is the same v7 one pr-labeler already uses. Contract suite passes under cozytest, and I mutated the workflow six ways (dropped the pr skip, flipped the dry_run default, added priority/backlog to signals, added contents: write, unpinned the action, moved cron before stale) and every one goes red, so the assertions bite.

Things I want changed:

  1. epic and lifecycle/frozen as accepted signals. I ran classify against the tree today: of the 43 issues that get triage/accepted, 9 are epic only and 5 are frozen only. Both labels are already on exempt-issue-labels, so they gain no exemption from this, the only effect is 14 issues leaving the needs-triage queue this PR exists to build. And labels.yml:109 says accepted means "ready to be actively worked on", frozen issue is not that. Drop them, or fix the description in labels.yml, but pick one. If you drop them the signal count goes to 3 and issue-triage-contract.bats:599 asserts -ge 4, so that bound goes down with it.

  2. Numbers in the description moved. It is 391 open issues and 248 untriaged now, not 277/138, because 118 issues were opened since you filed this. Split is 43 accepted / 205 needs-triage, 32 stale clocks reset, 18 issues get permanent stale exemption through the assignee only path. Still under the 400 cap, but those numbers are the justification.

  3. Sweep makes "open issue with no triage label" an unreachable state, remove one by hand and it comes back next morning. I am fine with that, but it is not what "the sweep never overwrites a decision someone made by hand" reads as, and nothing in the tree tells a maintainer it will happen. One line in the labeling bullet of AGENTS.md.

  4. Description says to run it by hand and read the summary, but the run only writes core.info, so a dry run over current backlog is 248 log lines. core.summary.addRaw(report).write().

Putting the contract under hack/ is what escalated this two file .github/ change to the full chainsaw suite, hack/select-e2e.sh:23 has hack/[^/]+\.bats$ in full_suite_pattern. Not for this PR to fix, just so it is a known cost on every later edit to that file. Also comment at the top of the bats names promote-gate-contract.bats and release-freeze-contract.bats, neither is on main yet.

Do you need the daily cron at all? The issues lane plus templates cover everything arriving from now on, and the backlog is a one time 248 issues that a single workflow_dispatch drains, then pacing, write cap and about half the script go away with it. If the cron is there to backstop hand removal then keep it, but that is point 3 and it is a policy call, not a repair.

Issues created via the API or gh bypass the issue templates and arrive
without any triage/* label; 252 of the 396 open issues carry none.
Label issues on arrival (opened/reopened) and run a daily sweep as a
backstop: issues already prioritised, assigned, or marked as epics get
triage/accepted, the rest triage/needs-triage.

A signal has to be on stale.yaml's exempt-issue-labels, because
triage/accepted is itself exempt and a signal that is not would grant a
permanent stale reprieve as a side effect of being labeled. That is
necessary and not sufficient: lifecycle/frozen is exempt and still not
a signal, because a lifecycle/* label says what the stale bot may do
with an issue and nothing about whether anybody reviewed it.

The sweep paces its writes and caps how many it makes per run, so a
pass over a large backlog stays inside GitHub's secondary rate limits
instead of failing partway through. Each run writes a job summary with
the counts, the issue numbers per label, and both remainders it can
leave behind, because a backlog pass logs a line per issue and nobody
reads 252 of those to find out what happened.

The manual dispatch defaults to dry-run, so the one entry point a human
can reach writes nothing until somebody unchecks the box. The schedule
is the one event that writes without being asked.

Deleting a triage label does not undo a decision: an issue left with
none is relabeled by the next sweep, so AGENTS.md's labeling section
says to change a decision by replacing the label instead.

No PR lane can exercise this workflow, so the first real run is a sweep
over every open issue. A contract test pins the executable lines that
bound that: the pull-request skip, the already-triaged skip, the pacing
and the cap, the dry-run default, the pinned action, the token scopes,
that no signal comes from the lifecycle/* namespace, and that the
summary is written with the await that makes it flush.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/issue-triage-contract.bats`:
- Around line 359-390: Extend the contract test “no lifecycle label sits in the
accepted-signal list” to also assert that the standalone `epic` label is absent
from `signals`, while preserving the existing non-empty guard and reliable
captured-status assertion pattern. Ensure the corresponding workflow
configuration removes `epic` from `ACCEPTED_SIGNAL_LABELS` unless the definition
of `triage/accepted` is intentionally changed.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8fa1c53a-b10d-45a6-928f-cdaad8462213

📥 Commits

Reviewing files that changed from the base of the PR and between e78bef9 and ea78d09.

📒 Files selected for processing (3)
  • .github/workflows/issue-triage.yaml
  • AGENTS.md
  • hack/issue-triage-contract.bats
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/issue-triage.yaml

Comment thread hack/issue-triage-contract.bats

@myasnikovdaniil myasnikovdaniil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Checked it by running, not by reading: pulled the script: block out and executed it under a mock against live listing of 576 issues, 39 accepted, 218 needs-triage, 319 skipped, 0 deferred, 257 writes against cap of 400. dryRun holds on every input I tried, including boolean true and string 'FALSE'. A 403/429 on fifth write stops after four and calls setFailed, a 404 goes on with failed=1.

lifecycle/frozen is out of the signal list and contract test now keeps the whole lifecycle/* namespace from getting back in, so five issues that had only frozen (#717, #679, #604, #154, #32) correctly get needs-triage. That was the only place where a wrong label was written. Rest of my earlier findings are advice, left inline.

Contract file is live, not decorative. 16/16 under hack/cozytest.sh, and I ran 20 mutations over a copy: dropped the pull-request skip, put lifecycle/frozen back, widened scope to contents: write, unpinned the action to @v7, moved cron before stale, and so on. Every one goes red on the assertion that owns it.

No ${{ }} interpolation in the file at all, payload is read as context.payload.issue, so injection through github.event.issue.* does not arise here. actions/github-script is pinned to the commit v7 points at, checked against API. Job permissions are issues: write and nothing else, still the narrowest of label-writing workflows in the tree. Both labels it can write are declared in labels.yml.

Nothing failed on ea78d09dd, several lanes are queued on runners since 14:11. Red run is on the superseded head dbc27e3f7: node-join on chainsaw/kubernetes-latest, known class from #3513, talos-image-cache still serving 206 for openstack-amd64.raw.xz at 18:30:07 when the 18m budget expired. Diff touches no e2e and no tenant-kubernetes path.

// cannot re-enter itself or wake any `labeled`-subscribed workflow.
//
// Adding a label bumps the issue's `updated_at`, which stale.yaml
// reads as fresh activity: the stale bot un-marks `lifecycle/stale`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking. This holds for 24 of the 32 issues currently carrying lifecycle/stale, but not for the other 8: #2527, #2222, #2116, #2094, #1961, #1960, #1561, #984 are assignee-only, so the sweep gives them triage/accepted, which is on exempt-issue-labels. In actions/stale at the digest stale.yaml pins, the exempt check returns at issues-processor.ts:386, before _removeStaleLabel at :834 is ever reached. So those 8 keep a stale badge nothing will remove, on issues nothing will close. Behaviour is harmless, but the comment overclaims for that slice, and it leaves 8 label removals of cleanup after the first sweep.

'priority/critical-urgent',
'priority/important-soon',
'priority/important-longterm',
'epic',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-derived at this head: 8 open issues have epic as their only accepted signal and get triage/accepted, that is #3761, #3278, #1266, #1262, #1261, #1247, #1246, #752. All of them roadmap trackers, while labels.yml reads triage/accepted as ready to be actively worked on. Not blocking: epic is maintainer-authored, it is already on exempt-issue-labels so no new reprieve is granted, and calling an epic needs-triage is the falser of the two readings. Recording the list because the count in the description drifted, and this is the set worth eyeballing once after the first sweep.

);
if (labels.some((n) => n.startsWith('triage/'))) return null;
const looked = // somebody has already looked at this one
(issue.assignees || []).length > 0 ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only signal that changes stale behaviour, and at this head it is 19, not 18: #3513, #3477, #3252, #3238, #3236, #2536, #2534, #2527, #2519, #2497, #2483, #2222, #2116, #2094, #1961, #1960, #1561, #984, #473. Checked each one, none carries an exempt-issue-labels label today, so triage/accepted is a genuinely new permanent reprieve in all 19. Escalation angle is closed, assignment needs write access and no issue template sets an assignee. Non-blocking, the list is here so it can be checked instead of taken on trust.

core.info(report);
// `write()` is async: unawaited, the step can end green with an
// empty summary panel. `addRaw` appends no newline of its own.
await core.summary.addRaw(report).write();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This closes my earlier note that the sweep reported only through core.info and a backlog dry run was 248 log lines. Summary is written, the await is there, addRaw gets the whole table including zero rows. Contract pins all three: dropping await, then deleting the ${deferred} row, then the ${unexamined} row, each turns the summary test red. Ran the block against live listing and the full table came out, so the first real sweep should read correctly too.

# contract. The workflow's own comments name every API and constant asserted
# below, so the JS block is filtered on `//` as well as `#`.
#
# No PR lane can exercise this workflow. It runs on `issues`, on a daily cron

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recording a cost this file takes on quietly. It matches hack/[^/]+\.bats$ in full_suite_pattern at hack/select-e2e.sh:85, and that rule is checked before inert_config_pattern makes .github/ inert. I fed this PR's three paths to hack/select-e2e.sh and it selects all 21 chainsaw suites. So from here every edit of this workflow and contract pair pays a full chainsaw run, which is what made this PR's own run 3h43m plus an unrelated node-join flake. Not this PR's to fix, the fix is either an exemption for contract-only bats files in select-e2e.sh or moving them out of the escalating glob.

}

# Minutes past midnight for a 5-field cron, from its first two fields.
cron_minutes() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cron_minutes reduces a cron to minutes past midnight, and the ordering test compares with -gt, which encodes later in the same UTC day rather than after. Correct for 04:37 against 05:53 and for any pair inside one day, but it inverts across midnight: put stale.yaml at 50 23 and this one at 30 0, ordering is still right and the test goes red at 30 against 1430. Not worth changing now, just the shape of the next false failure if either slot ever moves late.

@lexfrei
Aleksei Sviridkin (lexfrei) merged commit 61a236c into main Aug 12, 2026
19 of 23 checks passed
@lexfrei
Aleksei Sviridkin (lexfrei) deleted the feat/issue-triage-labeler branch August 12, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci Issues or PRs related to CI workflows, GitHub Actions, automation kind/feature Categorizes issue or PR as related to a new feature size/XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants