refactor(schema): one canonical container-format companion, not seven copies - #18
Conversation
… copies Every schematic shipped its own `SCHEMATIC.md.schema` — seven copies, 22 kB — of a document that describes the container format, not that package. Five were near-identical, one differed by a single See Also line, and two (`build-an-agent-dev-image`, `run-an-llm-router`) had been reworded and expanded, so the copies had already drifted: they disagreed about their own frontmatter comments, and only one carried the secrets, `*(observed)*`, dependency-pinning and artifact-path authoring rules. The format is versioned by the `spec:` field every SCHEMATIC.md carries in its frontmatter, so the companion now lives once at `schemas/spec-1/SCHEMATIC.md.schema`, keyed by that value. The canonical is the best of the seven rather than the oldest: it adopts the clearer header, the fuller section descriptions, the merged authoring rules and a `## Companion Files` section from the two expanded variants, and keeps the generic placeholder frontmatter example, the template pointer and the See Also block. No schematic's own name, date or path appears in it. `build-schematic` already had a canonical-fetch fallback; it now always uses it for SCHEMATIC.md, pointed at the new path instead of inside one arbitrary schematic. README and llms.txt state the split: the container-format companion is canonical, while artifact schemas (`agent.rego.schema`, `entrypoint.sh.schema`) stay alongside their file because they describe genuinely per-package content. The 18 artifact schemas are untouched. A dead pointer inherited from the old copy — `template/SCHEMA.template.md`, which exists nowhere in the repo — is replaced with the real template's path. Verified: `scripts/validate-catalog.sh` passes (19 entries); `find schematics -name '*.schema'` lists the 18 artifact schemas and no `SCHEMATIC.md.schema`; every remaining `SCHEMATIC.md.schema` reference resolves to the canonical path.
…aces The canonical companion paraphrased the section descriptions shorter than the two expanded copies it consolidates, dropping three specifics a reader of the format would have had: the Evidence table a reverse-engineered spec adds to its requirements, the Kind column (system | service | schematic) in the Dependencies table, and the ten binding principles kept verbatim with binding notes. Restored in the canonical's own voice; nothing else changes.
Deploying agentic-schematics with
|
| Latest commit: |
2bb0275
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://c5138ab8.agentic-schematics.pages.dev |
| Branch Preview URL: | https://refactor-canonical-format-sc.agentic-schematics.pages.dev |
phoenix-server
left a comment
There was a problem hiding this comment.
Reviewed by checking the deletion as well as the addition — a consolidation PR is a claim that nothing unique is being thrown away, so I tested that.
Nothing dangles. After the deletion: find schematics -name '*.schema' → 18 artifact schemas, zero SCHEMATIC.md.schema. No spec references a schema companion, no SCHEMATIC.md carries a $schema/yaml-language-server pointer that the deletion would strand, and nothing in the site assets, workflows, or the other skills names a deleted path — the only surviving references are the updated README, llms.txt, and build-schematic.
The canonical keeps the rules, and I found where it did not. Every authoring rule of the two expanded copies (build-an-agent-dev-image, run-an-llm-router) — secrets never appearing, *(observed)* for verified content, pin plus SHA-256 for schematic-kind dependencies, artifact-contract paths versus host paths, idempotency — is present in the canonical. But the section list was paraphrased shorter than the copies it replaces, and three specifics went missing: the Evidence table a reverse-engineered spec adds to its requirements, the Kind column (system | service | schematic) in the Dependencies table, and the ten binding principles kept verbatim with binding notes. I restored all three in the canonical's own voice (828abc9), so the PR now does what its body says it does — carry the fuller section descriptions, not a summary of them.
The skill change is the right one. build-schematic previously fell back to a canonical schema parked inside schematics/encrypt-container-secrets/; it now always uses schemas/spec-1/SCHEMATIC.md.schema and states that packages no longer ship a copy, which is correct — the format is keyed by the spec: frontmatter field, and each spec's own frontmatter still carries it. The dead template/SCHEMA.template.md pointer is replaced with the real skeleton, which exists (skills/schematics/skills/create-schematic/templates/SCHEMATIC.template.md).
Merges and validates against the post-#25 main. Clean merge; bash scripts/validate-catalog.sh → catalog ok: 19 entries, featured=[…], 18 specs, 21 pins verified, and the catalog check on this head passes.
There was a problem hiding this comment.
🟡 Changes recommended
Schema-version lookup and migration for existing schematics are incomplete.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Centralizes the SCHEMATIC.md format companion under schemas/spec-1/ and removes duplicated package-level copies.
Changes:
- Adds the canonical spec-1 schema.
- Updates build tooling and documentation references.
- Removes redundant package-local companions.
File summaries
| File | Change |
|---|---|
skills/schematics/skills/build-schematic/SKILL.md |
Updates canonical schema lookup. |
schematics/update-images-on-push/SCHEMATIC.md.schema |
Removes duplicate companion. |
schematics/run-an-llm-router/SCHEMATIC.md.schema |
Removes duplicate companion. |
schematics/restrict-docker-api-access/SCHEMATIC.md.schema |
Removes duplicate companion. |
schematics/expose-container-services-privately/SCHEMATIC.md.schema |
Removes duplicate companion. |
schematics/encrypt-container-secrets/SCHEMATIC.md.schema |
Removes duplicate companion. |
schematics/build-an-agent-dev-image/SCHEMATIC.md.schema |
Removes duplicate companion. |
schematics/authorize-docker-requests/SCHEMATIC.md.schema |
Removes duplicate companion. |
schemas/spec-1/SCHEMATIC.md.schema |
Adds canonical format companion. |
README.md |
Documents container and artifact schema separation. |
llms.txt |
Updates the canonical schema reference. |
Review details
- Files reviewed: 11/11 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ares The canonical companion is selected by the `spec:` frontmatter field, but 13 of the catalog's 18 specs declared no revision at all, and `build-schematic` fetched `schemas/spec-1/` unconditionally — a spec declaring a later revision would have been checked against revision 1 without a word. - add `spec: 1` to the 13 specs that lacked it - `build-schematic`: read the field, fetch `schemas/spec-<N>/`, and report a missing or unknown revision instead of assuming revision 1 - `validate-catalog.sh`: fail a spec with no `spec:` field, and one naming a revision the catalog has no companion for - schematics plugin 0.2.1 → 0.2.2 so installed copies pick up the skill change
|
Copilot's three comments are addressed in
On the head: |
There was a problem hiding this comment.
🟡 Changes recommended
The canonical schema rejects the repository’s published status, and validation mishandles inline comments on spec: 1; the README example also omits the required field.
Get a fresh assessment by requesting another Copilot review.
Review details
- Files reviewed: 26/26 changed files
- Comments generated: 3
- Review effort level: Lite
…atalog uses Copilot's re-review found the contract contradicting its own instances: - the canonical (and the create-schematic template it mirrors) allowed `draft | stable | superseded`, while 14 of the 18 specs carry `status: published` and the marketplace entry is what actually publishes them — a validator following the companion would have rejected the whole published catalog. `published` joins the allowed values, which is the side to fix: the catalog is the evidence, and the enum was the stale comment. - the new `spec:` check used `^spec:\s*(\S+)\s*\$`, which rejected the very line the template tells authors to write (`spec: 1 # schematic format version (this template's shape)`). An optional YAML comment is now allowed. - the README's frontmatter example omitted `spec:` (and `created`/`updated`), so copying it produced a spec the catalog check fails. schematics plugin 0.2.2 → 0.2.3 (the template changed).
|
Copilot's re-review is addressed in
|
…ed on no installable plugin (#19) Takes `schematics/authorize-docker-requests/` (0.3.0 → **0.4.0**) from "verified" to "deny-probed": the previous pass verified the decisions the author expected, and none of the ones an attacker would send. This pass adds the probes from #20 to the package's probe table first, shows them failing, then fixes the policy until they pass. Closes #20. ## What was actually broken A sandbox client could get **host root** in five ways, all of them a create request the policy treated as legitimate: - **A project-labelled create could ask for anything.** `HostConfig.Privileged: true` + `Binds: ["/:/host"]` + `PidMode: host` with `Labels: {com.docker.compose.project: backend-services}` → **allow**, and `POST /containers/<id>/start` is in the lifecycle allowlist for any container. Same with the testcontainers label, and with `CapAdd`, `Devices`, `SecurityOpt`, `UsernsMode`, or any bind outside the project directory. - **A project-named volume could be the host's root filesystem.** `POST /volumes/create {Name: "backend-services_root", DriverOpts: {type: none, o: bind, device: "/"}}` → **allow**; a container then mounts it, and the create-time gate (which reads `HostConfig`) sees nothing. - **A create-shaped body on `/containers/<id>/attach` satisfied the create rule.** The grant matched the path with `contains(...)`, so `POST /containers/<any>/attach` with `Body: {Name: "/buildx_buildkit_"}` → **allow**. `exec` → **allow** for the same reason. Only `POST /exec/<id>/start` was denied, so R-6 was not met and "Exec and attach are structurally excluded" was false. - **Network and volume deletes were unscoped**: `DELETE /volumes/hostdata`, `DELETE /networks/hostnet` → **allow**. - **A build could not work on a BuildKit daemon**: `POST /session` → **deny**, which is the first call the CLI makes (A-6's expected live failure). Plus the corrections the review asked for: `v0.10` embeds OPA **v1.3.0**, not v1.7.1; the Phase 6 `sed` commands failed on their own documented defaults; the header-as-list rule was dead; and the reverse-engineered input example put the container name in `Body`. ## What changed New requirements **R-15 … R-18**, each mapped to probe rows: | Req | Rule | |-----|------| | R-15 | Every create path (compose label, testcontainers label, volume create) passes a host-access gate: no `Privileged`, `CapAdd`, `Devices`, `SecurityOpt`, `VolumesFrom`, host or `container:`-joined `PidMode`/`IpcMode`/`NetworkMode`/`CgroupnsMode`, no `UsernsMode`, and no mount source that is neither a volume name nor a path inside `P-15` with no `..` segment. | | R-16 | A volume create may not carry `DriverOpts` and its driver must be `local` (or unset). | | R-17 | Grants match a path by **equality** (`/build`, `/images/create`, `/containers/create`), and no carve-out may be reachable from another endpoint. The BuildKit name-prefix carve-out is removed (P-11 retired); `POST /session` is granted (R-7 revision). | | R-18 | Network and volume deletes are scoped to the project as a whole path segment. | Every witness of a mount is checked — `Resolved` where the plugin supplies it, `Source`, and the raw `HostConfig.Binds`/`Mounts` — rather than the first that answers, so an inconsistent request refuses instead of passing. Also corrected, as requested: P-10 and the four other `1.7.1` claims (the version comes from the plugin's own `go.mod` at its tag); the Phase 6 `sed` uses `#` as its delimiter with `g`; the header-as-list rule and its comment are gone (Docker's `RequestHeaders` is `map[string]string`); the `Body.Name` example is replaced with the `name` query parameter; "structurally excluded" is replaced with the measured behaviour; and the `/workspace/containers/` references are gone (principles 2 and 3). The behavioural changes against `main` are more than two — an earlier body version said otherwise, and this is the corrected list: the project-scoped network and volume creates that were denied now work; path-named network/volume POSTs moved from a substring match (`contains(input.Path, "<P-3>")`, query string included) to a whole-segment match on `PathArr`; `BindMounts` is consulted on creates instead of only on path-named calls; grants match by equality instead of substring; network and volume deletes are now scoped; `POST /session` is granted; and every create now passes the R-15/R-16 gate. The first is a fix, the rest are tightenings. ## Verification The probe table in `skeleton/agent.rego.schema` grows from 25 rows to **71**, and each row is an `opa eval` against the substituted policy under the published `openpolicyagent/opa` images: ``` opa versions: 0.60.0 -> Version: 0.60.0; 1.3.0 -> Version: 1.3.0; 1.7.1 -> Version: 1.7.1 before: 71 probes | engines 3 | ROWS DEVIATE FROM EXPECTATION after: 71 probes | engines 3 | ALL ROWS AS EXPECTED ``` The 30 rows that decided wrong before, and are denied (or, for `/session`, allowed) now — on all three engines: ``` R15.01 Privileged + /:/host + PidMode host R15.14 CgroupnsMode host R15.02 BuildKit name + Privileged R15.15 bind of the docker socket R15.03 testcontainers label + Privileged R15.16 bind of /etc R15.04 attach with a smuggled BuildKit body R15.17 bind with a .. segment out of P-15 R15.05 exec with a smuggled BuildKit body R15.18 VolumesFrom a host container R15.06 CapAdd R15.25 Mounts bind outside P-15 R15.07 Devices R15.28 BindMounts enrichment outside P-15 R15.08 PidMode host R15.29 Resolved outside P-15 (symlink case) R15.09 IpcMode host R15.33 BindMounts, Resolved absent, outside P-15 R15.10 NetworkMode host R15.35 Mounts entry with no Type, outside P-15 R15.11 NetworkMode container:<id> R16.01 volume create DriverOpts bind of / R15.12 SecurityOpt seccomp=unconfined R16.05 volume create with a non-local driver R15.13 UsernsMode host R18.02 DELETE foreign volume R17.02 POST /session (deny → allow) R18.04 DELETE foreign network REG.05 /build/prune (allow → deny) R18.05 DELETE volume by opaque id ``` The other 41 rows decided identically before and after — the ordinary compose create, binds inside `P-15`, named volumes, testcontainers, lifecycle, read-only, host users, and both directions of the project-scoped create checks. One row (`R16.03`, a volume create with no `Driver` at all) failed on the first draft of R-16 and caught a Rego trap worth recording: `not is_string(field)` over a *missing* field fails the rule rather than succeeding, so presence tests now use `object.get(object, key, default)`. `bash scripts/validate-catalog.sh` → `catalog ok: 19 entries, featured=[...], 18 specs, 21 pins verified`. ## Deliberately not in this PR - **No live run.** Nothing here is deployed on any host; deploying this is root-only work. In particular `POST /session` was **not** measured against a live BuildKit daemon — no such host was available to this pass. Its grant rests on the client/daemon protocol, the probe row asserts only the policy's decision, and R-7 says so with an `inferred:` marker. A deployment whose daemon is BuildKit-disabled can drop that grant. - **Published host ports are outside the gate.** A project container may still map a host port; that is not root, but it can occupy a free port and answer for it. Closing it is daemon-configuration or firewall work, and Limitations now says so. - **Container delete and lifecycle calls stay unscoped** — the request carries nothing to attribute — and a network or volume delete by opaque id is now refused, so `docker volume rm <id>` stops working where `docker volume rm <project>_data` does not. Both are in Limitations. - **A symlink inside `P-15`** that points outside it survives where the plugin cannot read host paths (`Resolved` empty under a managed install). `..` segments are refused outright, so the residual case is a symlink. - **`improve-docker-security`'s R-2** ("the OPA policy narrows what the daemon *executes*") is not something this package enforces; it belongs in an issue against that schematic, and is not edited here. ## Merge order `#18` (one canonical container-format companion) is merged, and this branch is rebased onto it, so the per-package `SCHEMATIC.md.schema` — and its `template/SCHEMA.template.md` reference — is deleted in the base rather than resolved here. `main` is untouched. ## Reviewer's non-blocking points, answered - **Version level**: 0.4.0 (minor). R-15…R-18 are new requirements, which the update workflow calls a new capability; what narrowed is the policy's grants, which the requirements never intended to make. If a future change *removes* a requirement, that is the major bump. Recorded in Decisions rather than silently disagreed with. - **The in-place `Revision` notes** on R-5 and R-12 stay, with a dated revision note added under each: a reader needs the current requirement and its history in one place. - **No issue was referenced** — see `Closes #20` above. - **Snap confinement question** added as **Q-5**: whether a snap-confined daemon can bind-mount host `P-6` into the managed plugin at `/opa` is unverified here and needs a snap host to settle, exactly like Q-4 needs a maintenance window. ## Status Stays `draft`. No deployment, and the live acceptance tests (A-6, A-11, A-16) have not been run against a real daemon and policy. --------- Co-authored-by: phoenix <agent@phoenix.local>
…ons copied from frontmatter (#27) Closes #23. ## What was already resolved before this PR Two of the issue's findings no longer hold, both fixed by #18/#25: - **`status`:** the canonical companion `schemas/spec-1/SCHEMATIC.md.schema` now documents `draft | published | stable | superseded`, so the 14 `published` packages are compliant. No package changes status here, and no vocabulary is invented. - **`spec:`:** present in all 18 specs, and `scripts/validate-catalog.sh` fails when it is absent or names a missing companion. ## What this PR fixes 1. **`created` and `updated` on the 13 packages that omitted them.** Both are derived from history, not invented: `created` is the date of the first commit that added `schematics/<name>/`, `updated` the date of the most recent commit that touched it (`git log --diff-filter=A` / `git log -1`). They are frontmatter fields an author now maintains by hand, which is what the companion asks for. 2. **Description parity on 8 packages.** The companion says a catalog `description` is copied from the `SCHEMATIC.md` frontmatter, so the frontmatter wins and `.agent-schematics/marketplace.json` was synced to it. The differences are real edits (a `+` vs `and` in `encrypt-container-secrets`, em-dashes replaced by hyphens, a dropped clause, `every media arr` → `every media *arr`), not whitespace. The authoring plugin's own catalog entry is deliberately **not** touched: its `spec` field names `skills/create-schematic/SKILL.md`, a skill file with its own YAML `description`, so copying that into the plugin entry would have replaced a plugin description with a skill description. ## Verification ``` $ bash scripts/validate-catalog.sh catalog ok: 19 entries, featured=[...], 18 specs, 21 pins verified ``` Plus a direct check of the companion's REQUIRED fields and status enum across all 18 specs: **0 failing**. Executing it: apply a single expression to each pair. python3 with simple regexes to sweep 18 frontmatter blocks (name, version, status, spec, description, created, updated).
Cameri's observation, confirmed by measurement:
.schemacompanions are repetitive and bloat the repo. The whole of the problem isSCHEMATIC.md.schema— seven copies, 22,396 bytes — of a document that describes the container format, not the package it sits in.What the copies actually were
update-images-on-push,expose-container-services-privately,restrict-docker-api-accessencrypt-container-secretsrun-an-llm-router(4,498 B),build-an-agent-dev-image(4,698 B)So they had already drifted: the two expanded variants carried authoring rules the others lack — secrets never appearing at all,
*(observed)*for verified content,schematic-kind dependencies pinned to a commit plus a SHA-256, artifact-contract paths being legitimate where host paths are not — and the copies disagreed about their own frontmatter comments.The change
The format is versioned by the
spec:field everySCHEMATIC.mdcarries in its frontmatter, so the companion now lives once, atschemas/spec-1/SCHEMATIC.md.schema, keyed by that value.The canonical is the best of the seven, not the oldest — it adopts the clearer header, the fuller section descriptions, the merged authoring rules and a
## Companion Filessection from the two expanded variants, keeping the generic placeholder frontmatter example, the template pointer and the See Also block. No schematic's own name, date or path appears in it; the only dates are the placeholders the format example always had.build-schematicalready fetched a canonical schema as a fallback — it just pointed insideschematics/encrypt-container-secrets/, i.e. the canonical copy was itself parked in an arbitrary schematic. It now always uses the canonical forSCHEMATIC.md. README andllms.txtstate the split: the container-format companion is canonical, while artifact schemas (agent.rego.schema,entrypoint.sh.schema,compose.yml.schema) stay alongside their file, because they describe genuinely per-package content — the twoentrypoint.sh.schemafiles really are different documents.One dead pointer inherited from the old copy is fixed:
template/SCHEMA.template.mdexists nowhere in the repo; the real skeleton isskills/schematics/skills/create-schematic/templates/SCHEMATIC.template.md.Verification
bash scripts/validate-catalog.sh→catalog ok: 19 entries, rc=0find schematics -name '*.schema'→ the 18 artifact schemas, and zeroSCHEMATIC.md.schemagrep -rn "SCHEMATIC.md.schema" . --exclude-dir=.git→ every remaining hit resolves to the canonical path; none names a deleted fileAuthored by the
architectmember; reviewed and committed by the primary.Follow-ups on review
828abc9). The consolidation had paraphrased the section list shorter than the copies it replaced, so three specifics were missing: the Evidence table a reverse-engineered spec adds to its requirements, theKindcolumn (system | service | schematic) of the Dependencies table, and the ten binding principles kept verbatim with binding notes. Restored in the canonical's own voice.90862ff). The companion is selected by thespec:frontmatter field, but 13 of the catalog's 18 specs declared no revision at all, andbuild-schematicfetchedschemas/spec-1/unconditionally — a spec declaring a later revision would have been checked against revision 1 without a word. The 13 legacy specs now declarespec: 1; the skill reads the field and fetchesschemas/spec-<N>/SCHEMATIC.md.schema, reporting a missing or unknown revision as a deviation instead of assuming 1; andvalidate-catalog.shfails a spec with nospec:field, or one naming a revision with no companion inschemas/. Controls: with the migration reverted the check reports all 13;spec: 99reports the missing companion; deleting the line reports the first message again. No pin moves — a pin resolves at a commit sha, never the working tree.schematicsplugin goes 0.2.1 → 0.2.2, because this branch changesbuild-schematicand an installed copy only picks up a skill change on a version change.On the head
90862ff:catalog ok: 19 entries, featured=[…], 18 specs, 21 pins verified, with thecatalogand Pages checks green.2bb0275). The canonical alloweddraft | stable | supersededwhile 14 of the 18 specs carrystatus: published— a validator following the companion would have rejected the published catalog, sopublishedjoins the allowed values in the canonical and the template (the marketplace entry is what publishes a schematic; the enum was the stale comment). Thespec:check no longer rejects an inline YAML comment, which is what the template itself writes (spec: 1 # schematic format version …) — the old pattern returned no match on that line, the new one captures1. The README example gainedspec:,createdandupdated. Plugin 0.2.2 → 0.2.3.