test(conformance): see object kinds, and cover creating them from nothing - #195
Merged
Merged
Conversation
…hing
The suite verified diff -> apply -> compare, which is the right shape, but two
gaps meant whole classes of defect could not fail it.
**The fingerprint could not see most of a schema.** It compared columns,
constraints and indexes only. A partitioned table and an ordinary one are
indistinguishable in information_schema, which is how a migration that silently
flattened partitioning passed. It now also reads relkind and relpartbound,
views, triggers, routines, types with their enum labels, policies and sequence
parameters — the catalog, not the view over it.
Turning that on immediately failed 12 previously-green patterns. They are real:
for a column declared GENERATED BY DEFAULT AS IDENTITY with INCREMENT 10,
DBDiff emits only ALTER COLUMN ... TYPE integer and the identity is dropped
entirely. The old fingerprint could not notice because identity columns carry
no column_default.
**Every extracted pattern is an ALTER.** extract-patterns.php classifies
ALTER TABLE statements and explicitly skips PARTITION, INHERITS and REFERENCES,
so before_sql always creates a table and alter_sql mutates it. Nothing ever
built an object from nothing — which is precisely where the primary-key, serial,
enum and partitioning defects lived.
patterns-objects.json adds 20 hand-authored patterns with empty before_sql,
covering creation of tables with keys, serial and identity columns, enum,
composite and domain types, range and list partitioning, indexes on partitioned
parents, views, materialised views, functions, triggers (including on a
partitioned parent), policies, sequences, foreign keys between two new tables,
and generated columns.
14 pass. The 6 that do not are recorded, and the distinction between them
matters more than the count:
- 5 are unmodelled kinds — composite types, domains, materialised views,
policies and standalone sequences produce NO diff at all while the schemas
genuinely differ. A user sees "no drift" and believes their environments
match. That is the more dangerous class.
- 1 is wrong output — identity with non-default sequence options.
known-failures.json baselines all 18 by id and separates the two classes, so
the suite fails on new regressions rather than on this backlog. An entry that
starts passing is reported too, so the list cannot rot.
Also guards PDO::exec against an empty before_sql, which a create-from-nothing
pattern legitimately has.
Postgres suite 47 tests; unit + SQLite 334 tests.
Co-Authored-By: Claude <noreply@anthropic.com>
An identity column carries a sequence, and that sequence's options are part
of the column definition. The adapter read `is_identity` and
`identity_generation` but none of `identity_start`, `identity_increment`,
`identity_maximum`, `identity_minimum` or `identity_cycle`, so
id bigint GENERATED ALWAYS AS IDENTITY (INCREMENT 10 START 100)
was recreated as a plain identity counting up from one by one. The DDL
applies cleanly and the column looks right, so nothing fails — the next
insert just collides with rows the migration was meant to preserve.
Only non-default options are emitted. The defaults depend on the column's
type, and MAXVALUE for an `integer` identity is not the MAXVALUE for a
`bigint` one, so spelling out an inherited default would make every identity
type change read as two differences. Bounds are written out as literals
rather than computed: bigint's floor overflows a native PHP integer and
bcmath is not a dependency.
This was the single root cause behind every `wrong_output` entry in the
conformance baseline — 13 of them, including cases filed under alter_table
and change_type that did not look identity-related. `wrong_output` is now
empty and those entries are removed; the 5 remaining known failures are all
`unmodelled_kind` (composite types, domains, policies, sequences, matviews).
Verified against live PostgreSQL 16, 17 and 18: conformance exits 0 on all
three, mismatches 0. Full suite re-run — Unit 749, MySQL 17, Postgres 47,
SQLite 16 — since the bulk column query is shared and a change there is not
self-evidently Postgres-only.
Co-Authored-By: Claude <noreply@anthropic.com>
|
jasdeepkhalsa
added a commit
that referenced
this pull request
Aug 25, 2026
…formance (#196) Four separate definitions of "are these schemas the same?" had accumulated across this project and SupaForge, and they had drifted. This is the last of the four still carrying its own. Depends on [`@akalforge/pg-conformance@~0.0.1`](https://www.npmjs.com/package/@akalforge/pg-conformance). ## What the shared query adds The one it replaces could not see any of this: - view and materialised view **bodies** - routine **bodies** (it compared signatures only) - trigger **definitions** - identity sequence options - per-column generated / storage / compression / collation - constraint validity and deferrability - inheritance, and comments That blind spot is not theoretical here. The identity-options bug fixed in #195 sat in the baseline as **thirteen separate `wrong_output` entries** precisely because the fingerprint could not see sequence options and so mis-attributed one root cause across many patterns. Conformance still exits 0 on PostgreSQL 17 with the same **15 passes and 5 known unmodelled kinds**, so the extra strength costs nothing today. ## Better mismatch reports Reporting used to inspect only `$fp['columns']`, so a difference in a view body or a partition bound printed `schema mismatch` with nothing to point at. The fingerprint is now one entry per line, so the report names what actually differs: ``` expected: view public.v SELECT id FROM t WHERE id > 0; actual: view public.v SELECT id FROM t WHERE id < 0; ``` ## A silent failure guarded against `getSchemaFingerprint` treats a missing `fingerprint` column as a fault rather than as "no differences". An empty fingerprint would make **every** comparison succeed, and a fully green run would be indistinguishable from a correct one. ## Why npm and not Composer The package ships a PHP accessor, so `composer.json` is untouched and no registry credentials are involved — the conformance job runs `npm ci` and requires the file by path. DBDiff uses node for nothing else, so `package.json` is explicitly test-only tooling. Pinned `~0.0.1`, not `^0.0.1`: under semver a caret on a `0.0.z` version permits no updates at all, and `npm install` silently rewrites a tilde to a caret via its default `save-prefix`. ## Verification Run against live PostgreSQL 17, and the full suite re-run because the harness is shared rather than Postgres-only: | suite | result | | --- | --- | | Unit | 749 tests | | MySQL | 17 tests | | Postgres | 47 tests | | SQLite | 16 tests (1 skipped) | | PG conformance | exit 0 — 15 pass, 5 known | 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Two gaps, both of which let whole classes of defect pass
The suite already does the right thing — extract patterns from PostgreSQL's own regress files, diff, apply, and verify the result matches. That last step is the part most tools skip. But two gaps meant real bugs could not fail it.
1. The fingerprint could not see most of a schema
getSchemaFingerprint()compared columns, constraints and indexes — nothing else. A partitioned table and an ordinary one are indistinguishable ininformation_schema, so a migration that silently flattened partitioning passed cleanly.It now reads the catalog rather than the view over it:
relkindandrelpartbound, views, triggers (excluding the copies PostgreSQL clones onto partitions), routines, types with their enum labels in sort order, policies, and sequence parameters.Turning that on immediately failed 12 previously-green patterns. They are real. For a column declared
GENERATED BY DEFAULT AS IDENTITYwithINCREMENT 10:The identity is dropped entirely. The old fingerprint could not notice, because identity columns carry no
column_default.2. Every extracted pattern is an ALTER
extract-patterns.phpclassifiesALTER TABLEstatements and explicitly skipsPARTITION,INHERITSandREFERENCES. Sobefore_sqlalways creates a table andalter_sqlmutates it — nothing ever builds an object from nothing.That is exactly where the primary-key, serial, enum and partitioning defects lived, which is why 336 patterns and a convergence check never caught them.
What this adds
patterns-objects.json— 20 hand-authored patterns with emptybefore_sql, covering creation of: tables with keys, serial and identity columns, enum / composite / domain types, range and list partitioning, indexes on partitioned parents, views, materialised views, functions, triggers (including on a partitioned parent), policies, sequences, foreign keys between two new tables, and generated columns.14 pass. The 6 that do not split into two classes, and the distinction matters more than the count:
The unmodelled ones are the dangerous class: a user sees "no drift" and believes their environments match.
Keeping the suite usable
known-failures.jsonbaselines all 18 by id, separated by class, so the suite fails on new regressions rather than on this backlog. An entry that starts passing is reported too, so the list cannot rot silently:Also guards
PDO::execagainst an emptybefore_sql, which a create-from-nothing pattern legitimately has.Verification
🤖 Generated with Claude Code