Skip to content

catalog: re-pin dependency links to a reachable commit; verify pins in CI - #25

Merged
phoenix-server merged 4 commits into
mainfrom
claude/schematics-pr-review-85jayq
Sep 17, 2026
Merged

phoenix-server merged 4 commits into
mainfrom
claude/schematics-pr-review-85jayq

Conversation

@cameri

@cameri cameri commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Refs #21 (dead pins) and #22 (nothing runs the validator).

Problem

Every schematic-kind dependency pin in the catalog (21 across 8 packages) points at blob/56e02f9/…. That commit is not in this repository's history, so every raw URL 404s and no composition's A-7 ("fetch at the pinned commit, hash, compare") can pass. Six of the 21 hashes match no file content that has ever been on main. scripts/validate-catalog.sh did not look at pins, and nothing ran it anyway.

Change

Pins. All 21 re-pointed to 81721d8ff548ad0f4b1477e696b7899d30f999fa, the commit whose file contents the 15 surviving hashes were computed from (every pinned file is unchanged since). Those 15 keep their hash; the link is the only thing that changed.

The six unverifiable ones (three in run-a-movies-and-series-library, three in run-a-music-library) are refreshed to the sha256 of the published content at that commit. The pre-rewrite content is unrecoverable, so this is the only state in which A-7 can be run at all. Each composition's Decisions section records the re-pin and names the three contracts, both bump to 0.2.2, and the semantic re-read of those six contracts stays open as the checklist in #21. The other six files get a one-line Decisions entry and no version change (link fix, contents identical).

Validator. scripts/validate-catalog.sh now also checks:

  • every pin: commit exists in this repository, file exists at that commit, sha256 matches; no blob/main|HEAD|master/ links
  • every modules/, scripts/, skeleton/, templates/, assets/ path a spec references exists in its package
  • every composes[] entry names a catalog plugin
  • fixes the source prefix strip (lstrip('./') strips characters, not the prefix)

A shallow or non-git checkout skips the pin checks with a warning rather than reporting false failures.

CI. .github/workflows/validate.yml runs the script on every pull request and push to main, with fetch-depth: 0 so pins resolve. No deploy step; Cloudflare keeps that.

Verification

$ bash scripts/validate-catalog.sh
catalog ok: 19 entries, featured=[…], 18 specs, 21 pins verified

Negative tests, each restored afterward: one pin pointed back at 56e02f9FAIL: pinned commit 56e02f9 is not in this repository; one hash corrupted → FAIL: sha256 mismatch … pinned 0000b9b1d34d, actual a776b9b1d34d; one link changed to blob/main/FAIL: dependency link uses floating ref 'main'. Run against main's specs, the new validator reports all 21 pins as unreachable.

Not in this PR: frontmatter/status/description-parity checks (#23), which would fail on main today until the content is fixed.

Note for #19: improve-docker-security's D-3 now pins the v0.2.1 commit, which stays valid after #19 merges; a deliberate re-pin to 0.3.0 is a follow-up.

🤖 Generated with Claude Code

https://claude.ai/code/session_011mP1bMVBfB4GY5hsZXySwB


Generated by Claude Code

…n CI

Every schematic-kind dependency pin in the catalog (21 across 8 packages)
pointed at commit 56e02f9, which is not in this repository's history, so
no composition's A-7 hash check could pass and the raw URLs 404.

- Re-point all pins to 81721d8, the commit whose file contents the
  surviving hashes were computed from. 15 hashes are unchanged.
- Six pins in run-a-movies-and-series-library and run-a-music-library
  carried hashes that match no content ever published on main; refresh
  them to the published contents at 81721d8, log the re-pin in each
  Decisions section, and bump both compositions to 0.2.2. The semantic
  re-read of those six contracts stays open in #21.
- Extend scripts/validate-catalog.sh: pinned commit exists, file exists
  at that commit, sha256 matches, no floating refs; every modules/,
  scripts/, skeleton/, templates/, assets/ path a spec references
  exists; every composes[] entry names a catalog plugin. Fix the
  source-prefix strip (lstrip removed characters, not the ./ prefix).
- Add .github/workflows/validate.yml so the script runs on every pull
  request and push to main with full history (fetch-depth: 0).

Refs #21, #22.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011mP1bMVBfB4GY5hsZXySwB
Copilot AI lite review requested due to automatic review settings September 17, 2026 04:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical validator and CI issues remain, and the required semantic re-reads are still open.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR repairs 21 schematic dependency pins and adds catalog validation in CI.

Changes:

  • Re-pins dependencies to a reachable commit and refreshes six hashes.
  • Adds checks for pins, referenced files, and composition entries.
  • Runs validation on pull requests and pushes to main.
File summaries
File Reviewed changes and findings
scripts/validate-catalog.sh Adds catalog and pin validation. Critical (3 votes): malformed or non-commit schematic pins can be accepted or ignored. Moderate (1 vote): paths are not constrained to schematic contracts; commit reachability is not verified. Moderate (2 votes): supported paths embedded outside immediate backticks may be missed. Nit (3 votes): skipped pin checks are still reported as verified.
schematics/run-an-llm-router/SCHEMATIC.md Re-pins the encrypted-secrets dependency.
schematics/run-a-music-library/SCHEMATIC.md Re-pins dependencies and bumps the version. Moderate (3 votes): three refreshed dependencies have an open semantic re-read while version 0.2.2 is published.
schematics/run-a-movies-and-series-library/SCHEMATIC.md Re-pins dependencies and bumps the version. Moderate (3 votes): three refreshed dependencies have an open semantic re-read while version 0.2.2 is published.
schematics/run-a-book-library/SCHEMATIC.md Re-pins dependencies.
schematics/improve-docker-security/SCHEMATIC.md Re-pins dependencies.
schematics/fetch-series-over-usenet/SCHEMATIC.md Re-pins the shared dependency.
schematics/fetch-music-over-usenet/SCHEMATIC.md Re-pins the shared dependency.
schematics/fetch-movies-over-usenet/SCHEMATIC.md Re-pins the shared dependency.
.github/workflows/validate.yml Runs validation with full history. Critical (1 vote): workflow permissions and persisted checkout credentials allow PR-controlled code to retain write-capable repository access.
Review details

Suppressed comments (2)

scripts/validate-catalog.sh:61

  • The captured path is unconstrained beyond whitespace/parentheses, so a Kind=schematic dependency could point to schematics/foo/README.md (or another existing file) and pass the existence/hash checks. The composition convention requires the dependency's schematics/<name>/SCHEMATIC.md; validate that path shape instead of treating any hashed repository file as a schematic contract.
PIN = re.compile(r'https://github\.com/cameri/schematics/blob/([^/\s)]+)/([^)\s]+)\)\s*`sha256:([0-9a-f]{64})`')

scripts/validate-catalog.sh:83

  • git cat-file -e <commit>^{commit} only proves that the object is present in the local object database; it does not prove that the commit is reachable from repository history. A dangling commit can therefore pass the “in this repository” check. Check ancestry/reachability from the repository refs before validating the file.
        if git('cat-file', '-e', commit + '^{commit}').returncode != 0:
  • Files reviewed: 10/10 changed files
  • Comments generated: 6
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/validate.yml
Comment thread scripts/validate-catalog.sh Outdated
Comment thread schematics/run-a-movies-and-series-library/SCHEMATIC.md Outdated
Comment thread schematics/run-a-music-library/SCHEMATIC.md Outdated
Comment thread scripts/validate-catalog.sh Outdated
Comment thread scripts/validate-catalog.sh Outdated
Address the review on #25:

- Workflow: permissions contents: read and persist-credentials: false,
  so the validator runs PR-controlled code without a writable token.
- Pins: any link into this repository's blob/ tree must be a
  well-formed pin ([<name> v<version>](.../blob/<sha>/schematics/<name>/
  SCHEMATIC.md) `sha256:<64 hex>`); anything else on the line is a
  malformed-pin failure instead of being ignored. The ref must be a hex
  commit id that resolves to a commit (a tag or branch name is
  rejected even when it points at the right commit), be reachable from
  the checked-out history (a dangling object is rejected), and the
  link's name and version must match the pinned file's directory and
  frontmatter.
- Referenced files: match package paths anywhere in the spec, not only
  directly after a backtick.
- Report "pins found, not verified" when verification was skipped.
- run-a-movies-and-series-library, run-a-music-library: the semantic
  re-read of the six re-pinned contracts is done and recorded in each
  Decisions entry. Music: no drift. Movies: Phase 0 still named D-2 and
  D-3 only from before the fetching half was split into three packages;
  it now orders all four dependencies.

Refs #21, #22.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011mP1bMVBfB4GY5hsZXySwB
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 17, 2026

Copy link
Copy Markdown

Deploying agentic-schematics with  Cloudflare Pages  Cloudflare Pages

Latest commit: 991fe2f
Status: ✅  Deploy successful!
Preview URL: https://f6fc1e86.agentic-schematics.pages.dev
Branch Preview URL: https://claude-schematics-pr-review.agentic-schematics.pages.dev

View logs

Claude and others added 2 commits September 17, 2026 11:23
Rewrite the eight 2026-09-17 Decisions entries so they tell a builder
what the pin is and how to verify it (raw file at the pinned commit,
sha256sum, compare with the table), and, for the two compositions, the
concrete cross-references into their dependencies that the glue relies
on. Verification is stated in the self-contained form the composition
convention prescribes rather than by pointing at a repository script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011mP1bMVBfB4GY5hsZXySwB
…a local reference

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011mP1bMVBfB4GY5hsZXySwB

@phoenix-server phoenix-server left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by running it, not reading it — the whole PR is a claim about reachability, so I checked reachability.

The defect is real, and the control shows it. Copying this PR's scripts/validate-catalog.sh onto main's tree fails on every pin: pinned commit 56e02f9 is not a commit in this repository — the 21 dead links described, reproduced independently.

The new checks bite. At the PR head the validator passes (catalog ok: 19 entries, featured=[…], 18 specs, 21 pins verified), and each guard was broken deliberately and restored:

  • digest corrupted → FAIL … sha256 mismatch … pinned 000000000000, actual 86cffb26f187
  • link changed to blob/main/FAIL … pin uses ref 'main'; pin a commit sha, not a tag or branch
  • digest removed from a pin → FAIL … malformed dependency pin
  • a modules/does-not-exist.md reference appended → FAIL … references modules/does-not-exist.md, which does not exist in the package

The re-pin is to content, not to a green check. 81721d8 is an ancestor of main; the digest of each pinned file at that commit is what the table now carries, and the validator verifies ancestry, existence at the commit, sha256, and that the pinned file's own version: matches the version in the link text. I spot-checked the semantic re-read recorded in the two compositions' Decisions entries against the newly pinned files: fetch-music-over-usenet P-1 is ${MEDIA_ROOT}/music, P-2 is MUSIC_CATEGORY; fetch-over-usenet P-6 carries audio=3000; run-a-movies-and-series-library composes fetch-over-usenet and serve-movies-and-series — which is what those entries claim, so the cross-references the glue depends on do resolve at the new pin. The shared-downloader wording fix in 991fe2f is correct too: it was a local modules/ path that no package contains, now named as the module in its owning package.

CI. Workflow-level contents: read, persist-credentials: false, fetch-depth: 0; PR-triggered code gets no push capability.

One non-blocking observation for the record: rewriting the eight Decisions entries dropped the #21 provenance sentence ("re-pointed from 56e02f9, which is not in this repository's history") — the issue reference now lives only in this PR body and in the inline review replies, so a reader of main cannot see why the pins moved, which is the kind of thing a living spec is supposed to carry. Not worth another commit here; it belongs with the #21 close-out.

@phoenix-server
phoenix-server merged commit b7f74c9 into main Sep 17, 2026
2 checks passed
@phoenix-server
phoenix-server deleted the claude/schematics-pr-review-85jayq branch September 17, 2026 12:26
phoenix-server added a commit that referenced this pull request Sep 17, 2026
…#29)

Closes #22 — proposal 3 is the last piece.

## What was already fixed before this PR

Proposals 1 and 2 landed in #25:

- `.github/workflows/validate.yml` runs `bash
scripts/validate-catalog.sh` on every `pull_request` and on `push` to
`main`, with `fetch-depth: 0` and a read-only token, no deploy step.
- The validator was extended with exactly the checks the issue named:
every `schematic`-kind pin resolves (commit reachable from history, file
present at it, sha256 matches, pin text's version equals the linked
file's frontmatter), every
`modules/`/`scripts/`/`skeleton/`/`templates/`/`assets/` path a spec
references exists in the package, and every `composes[]` name exists in
the catalog. Run on `main`: `catalog ok: 19 entries, featured=[...], 18
specs, 21 pins verified`.

## What this PR adds

Proposal 3, the stronger of the two options: the rule is now enforced
instead of softened.

- `.github/workflows/require-issue.yml` fails a pull request whose body
references no issue (`#N`), with a check message that cites rule zero
and links `CONTRIBUTING.md`.
- The job is exempt for `OWNER`, `MEMBER`, `COLLABORATOR` and bots. Rule
zero exists so that an outside contributor's idea gets a public
discussion before anyone spends effort; someone with push access does
not need that gate, and a bot cannot open an issue.
- `CONTRIBUTING.md` now names the workflow and states the exemption, so
the promise and the automation say the same thing.

## Note on proposal 4

Issue #8 (GitHub Pages previews) is superseded, not pending: Cloudflare
Pages deploys every PR and comments the preview URL — see the bot
comment and the passing `Cloudflare Pages` check on #19. #8 can be
closed.

## What is proven live, and what is not

The **skip** path is proven on this PR: the `reference` check appears
and reports `skipping`, because the branch was pushed by a collaborator.
The **fail** path is proven only as a shell test (`no issue link` → exit
1, pasted above): exercising it on GitHub needs a pull request from an
account that is not a collaborator, which cannot be arranged from here.
Worth a one-off check the next time an outside contribution arrives.
phoenix-server added a commit that referenced this pull request Sep 17, 2026
…ons copied from frontmatter (#27)

Closes #23.

## What was already resolved before this PR

Two of the issue's findings no longer hold, both fixed by #18/#25:

- **`status`:** the canonical companion
`schemas/spec-1/SCHEMATIC.md.schema` now documents `draft | published |
stable | superseded`, so the 14 `published` packages are compliant. No
package changes status here, and no vocabulary is invented.
- **`spec:`:** present in all 18 specs, and
`scripts/validate-catalog.sh` fails when it is absent or names a missing
companion.

## What this PR fixes

1. **`created` and `updated` on the 13 packages that omitted them.**
Both are derived from history, not invented: `created` is the date of
the first commit that added `schematics/<name>/`, `updated` the date of
the most recent commit that touched it (`git log --diff-filter=A` / `git
log -1`). They are frontmatter fields an author now maintains by hand,
which is what the companion asks for.
2. **Description parity on 8 packages.** The companion says a catalog
`description` is copied from the `SCHEMATIC.md` frontmatter, so the
frontmatter wins and `.agent-schematics/marketplace.json` was synced to
it. The differences are real edits (a `+` vs `and` in
`encrypt-container-secrets`, em-dashes replaced by hyphens, a dropped
clause, `every media arr` → `every media *arr`), not whitespace.

The authoring plugin's own catalog entry is deliberately **not**
touched: its `spec` field names `skills/create-schematic/SKILL.md`, a
skill file with its own YAML `description`, so copying that into the
plugin entry would have replaced a plugin description with a skill
description.

## Verification

```
$ bash scripts/validate-catalog.sh
catalog ok: 19 entries, featured=[...], 18 specs, 21 pins verified
```

Plus a direct check of the companion's REQUIRED fields and status enum
across all 18 specs: **0 failing**.

Executing it: apply a single expression to each pair. python3 with
simple regexes to sweep 18 frontmatter blocks (name, version, status,
spec, description, created, updated).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants