Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
546b17a
Core vertical slice: docs/abi.md contract, native/lgj-abi, Java facade
AdaWorldAPI Aug 17, 2026
a1e65cc
Merge main (bootstrap commit) into dev branch to establish shared his…
AdaWorldAPI Aug 17, 2026
bdd40b2
Valhalla lab: three-truths method, causal isolation, 3 real reproducers
AdaWorldAPI Aug 17, 2026
317ee83
Merge main (squash of PR #1) to fix history divergence for PR #2
AdaWorldAPI Aug 17, 2026
ea4753c
Merge main (squash of PR #2) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
4ab0da8
Vector API bench: real JMH, cross-checked; the crossing does not alwa…
AdaWorldAPI Aug 17, 2026
96eccff
Merge main (squash of PR #3) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
cd7782b
Phase I synthesis docs + fusion-sweep 256-row re-run + board hygiene
AdaWorldAPI Aug 17, 2026
ddeaecb
Merge main (squash of PR #4) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
e4463cb
Board: PR #4 arc entry (post-merge)
AdaWorldAPI Aug 17, 2026
e8b6dc2
SoA row store: 512B rows, 32 facet lanes, ABI minor 2 (W1+W2)
AdaWorldAPI Aug 17, 2026
aff72f3
Merge main (squash of PR #5) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
8fceaca
Board: PR #5 arc entry (post-merge)
AdaWorldAPI Aug 17, 2026
91e22f3
Knowledge: assess the archived layout-bridge discussion; name W6
AdaWorldAPI Aug 17, 2026
6927961
Plans: OGAR Machine (exploratory) + lance-graph-hydrate dependency note
AdaWorldAPI Aug 17, 2026
7e218ed
Merge main (squash of PR #6) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
9cc85ad
Board: PR #6 arc entry (post-merge)
AdaWorldAPI Aug 17, 2026
f3bbab7
Waves calcified: dispatch maps for every plan; Ghidra plan from real …
AdaWorldAPI Aug 17, 2026
dac0f0b
Merge main (squash of PR #7) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
57a7e48
Board: PR #7 arc entry (post-merge)
AdaWorldAPI Aug 17, 2026
83b8994
Plan: lance-graph #958 merged (was open at last check)
AdaWorldAPI Aug 17, 2026
572f865
Java RowStore facade: W3 shipped (185/185, one bug found+fixed)
AdaWorldAPI Aug 17, 2026
b138382
Merge main (squash of PR #8) to keep dev branch history aligned
AdaWorldAPI Aug 17, 2026
bc6f91a
Board: PR #8 arc entry (post-merge)
AdaWorldAPI Aug 17, 2026
619b3de
Plan: r2sleigh recorded as third lift path + decompiler candidate
AdaWorldAPI Aug 17, 2026
84e73ca
Bench Component F: the boundary re-asked on the real row-store layout…
AdaWorldAPI Aug 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@
## 2026-08-17 (dispatch 2) — W4 measured: the boundary re-asked on the REAL layout

`wave-substrate-w3-w4.md` Dispatch 2 executed: one Sonnet worker
(Component F: `F_RowStoreFacetScan` + `RowStoreData` + the two
`Kernels` facet-match arms, mirroring the Rust kernel's chunk algorithm
line-for-line incl. the `& 0x1111` classid-position mask), orchestrator-run
JMH, 9/9 combos, the cross-check green at every row count before anything
was timed.

**The finding: Component C's direction survives; its margin collapses.**
The Vector API still wins the per-row 32-facet scan at every row count —
but by **2.51× / 1.92× / 1.14×** (4K / 65K / 1M rows) against C's 56×, and
at 512 MiB traversed all three arms converge on memory bandwidth
(~6–7 GB/s on this container). More work per byte narrows the boundary
exactly as `execution-boundary.md` predicted; it now says so as
measurement. One disclosed asymmetry: the native arm allocates its output
per call (`facetMatchesInto` named as the follow-up if the small-row gap
ever matters).

Also: `summarise.sh` gained the F table (and its old "E/F" section title —
a real collision with the new component — was corrected to "E");
`TABLES.md` regenerated from the merged CSV; `RESULTS.md` §F written;
`RowStore` gained a package-private `handle()` (mirroring
`NativePattern`'s, for the bench's split-package `NativeAccess` bridge
only). Substrate wave file fully executed — both dispatches shipped.

## 2026-08-17 (dispatch 1) — W3 shipped: the Java `RowStore` facade, from the calcified wave map

First real dispatch of the wave system: `wave-substrate-w3-w4.md` Dispatch 1
Expand Down
20 changes: 20 additions & 0 deletions .claude/board/PR_ARC_INVENTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,26 @@
> anti-pattern the imported board rules name. Backfilled below in one
> pass rather than left stale; PR #4 onward gets its entry at merge time.

## PR #8 — Java RowStore facade: W3 shipped (merged 2026-08-17, squash `320808d`)

- **Added:** `RowStore`/`FacetMatchView`/`FacetId`/`NativeResource` public
API; `Mask.source()` retyped `NativePattern → NativeResource`;
`RowStoreParityTest`/`RowStoreLifetimeTest` (53 new checks).
- **Locked:** the wave-dispatch system works end to end — 3 disjoint
Sonnet workers, zero merge conflicts, mutually consistent signatures
with no coordination beyond the frozen briefs
(`E-LGJ-WAVE-DISPATCH-VALIDATED-1`).
- **Deferred:** W4 (bench Component F) — the wave file's second dispatch.
- **Docs:** `STATUS_BOARD` D-LGJ-W3 DONE; `LATEST_STATE`; EPIPHANIES entry
incl. an orchestrator-side false alarm (guessed env var name instead
of reading `Abi.java`'s `ENV_LIBRARY` constant) recorded so it isn't
repeated.
- **Confidence:** High — 185/185 (was 132), 0 new lint warnings, one real
bug (`FacetMatchView.rowCount()` missing its closed-store guard) caught
by the mandated tests and fixed before merge, both disable-runs
red-then-green with the exact predicted blast radius. Bot reviewers at
usage limits, did not run.

## PR #7 — waves calcified: dispatch maps for every plan (merged 2026-08-17, squash `68f7add`)

- **Added:** `.claude/waves/` — README (standing rules + verbatim worker
Expand Down
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,5 @@ layout wired end to end. Doctrine: `E-LGJ-THE-MIDDLE-TIER-IS-DELETED-NOT-WRAPPED
| D-LGJ-W1 | ndarray: `MultiLaneColumn::iter_u32x16`/`len_u32x16` + `eq_u32_strided_to_mask` (W1a contract) | **DONE 2026-08-17** — ndarray PR #279; `simd_int_ops` 46/46 (5 new strided tests incl. two `should_panic` bounds/overflow arms + stride-4 parity against the contiguous primitive), `simd_soa` 15/15, full `simd` 263/263, doctests, clippy `-D warnings` + fmt clean |
| D-LGJ-W2 | lgj-abi row store: `rowstore.rs`, `LGJ_RESOURCE_ROWSTORE`, `lgj_rowstore_open`, strided facet lanes through the unchanged `LgjLaneDesc`, `lgj_op_eq_classid`, `lgj_row_facet_match`, ABI minor 1→2, `docs/abi.md` §11 | **DONE 2026-08-17** — `cargo test` **84/84**, clippy/fmt clean, release build exports **18/18** symbols (`nm -D`). Parity: both kernels vs independent scalar references over 10 row counts × 2 seeds × 4 facets × 4 needles, cross-checked a THIRD way against `RowStore::classid_at`. Two-sided payload-vs-classid falsifier. End-to-end membrane test covers describe → predicate → mask algebra → count → facet-match → lifecycle |
| D-LGJ-W3 | Java `RowStore` facade: structured `MemoryLayout`, minor-≥2 gate, `FacetMatchView`, parity test transcribing the generator | **DONE 2026-08-17** — dispatched per `.claude/waves/wave-substrate-w3-w4.md` (3 Sonnet workers, disjoint scopes: FFM membrane extension / public facade / tests), orchestrator-integrated. `javac -Xlint:all` clean (same 7 pre-existing `[restricted]` warnings, zero new). `AllTests` **185/185** (was 132; +53 new checks: 29 parity + 24 lifetime). **One real bug caught by the suite and fixed**: `FacetMatchView.rowCount()` was missing the closed-store guard `matchesOf`/`cardinality` both had — a stale row count was readable after the owning store closed. Fixed, re-verified. Both mandated disable-runs ran red-then-green: (1) `Abi.requireMinor` inflated by 1 → exactly `RowStoreParityTest`+`RowStoreLifetimeTest` failed, all 8 other suites stayed green; (2) the pure-Java generator's a/b draw order swapped in `RowStoreParityTest` → exactly that suite broke (17/29), `RowStoreLifetimeTest` (generator-independent) stayed green — confirming the parity test is a real falsifier, not decorative. `Mask.source()` retyped `NativePattern → NativeResource` (new interface) so a `Mask` can parent onto either a `NativePattern` or a `RowStore` — zero call-site breakage (verified: no existing caller bound the narrower type) |
| D-LGJ-W4 | Bench Component F: Vector API facet scan vs the crossing, on the REAL layout | Queued |
| D-LGJ-W4 | Bench Component F: Vector API facet scan vs the crossing, on the REAL layout | **DONE 2026-08-17** — 1 Sonnet worker (F_RowStoreFacetScan + RowStoreData + Kernels facet-match arms, cross-check-in-@Setup discipline), orchestrator-run JMH: 9/9 combos, cross-checks green at every row count. **Finding: Component C's direction survives, its margin collapses** — Vector API wins the 32-facet strided scan at every row count but by 2.51×/1.92×/1.14× (4K/65K/1M rows) vs C's 56×; at 512 MiB traversed all three arms converge on memory bandwidth. Native arm's per-call allocation asymmetry disclosed in §F with a named follow-up (`facetMatchesInto`), not hidden. summarise.sh extended with the F table (and the old 'E/F' section retitled 'E' — a real naming collision); tables regenerated from the merged CSV |
| D-LGJ-W5 | Three consumer examples (trades / bricks / graph) — one plan file each | Planned, gated on W3 |
37 changes: 37 additions & 0 deletions .claude/plans/ghidra-integration-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,43 @@ License note: Ghidra core is Apache-2.0 (compatible with everything in
this stack); the `GPL/` subtree (demangler etc.) is not needed for
lifting and stays untouched.

## G0 addendum (2026-08-17, operator-flagged): `AdaWorldAPI/r2sleigh` — the third lift path and the decompiler candidate

Read-only clone at `/workspace/adaworldapi/r2sleigh` (HEAD `60942f6`, 20 MB,
Rust 1.93 workspace, 8 crates + an r2 plugin). What it is, verified from the
tree, not the README alone:

- **Pipeline:** `.sla (Ghidra processor spec) → libsla → P-code → r2il
(typed IR, 60+ opcodes) → {SSA (r2ssa) → taint/symbolic (r2sym, Z3-backed),
decompiler-to-C (r2dec), type inference (r2types), ESIL}`.
- **The honest FFI fact:** `Cargo.lock` carries `libsla` AND `libsla-sys` —
the SLEIGH runtime underneath is **Ghidra's native C++ via FFI**, not a
pure-Rust SLEIGH. r2sleigh's own crates are Rust; the lifter core is not.
Acceptable on exactly the same footing as running Ghidra itself: this is
a LIFT-TIME tool, never at OGAR-Machine runtime (the no-C-at-runtime rule
is about the execution path, and lift time is not on it).

**Consequences for the waves:**

1. **G1 gains a candidate C: lift via `r2sleigh-cli`** (Rust CLI consuming
Ghidra's own `.sla` specs) alongside A (released-Ghidra
`analyzeHeadless`) and B (fork build). No JVM in the lift loop, and the
P-code comes from the SAME SLEIGH specs Ghidra uses — so the G1
falsifier gets stronger, not weaker: r2sleigh's P-code dump vs Ghidra's
own listing is a **cross-implementation** agreement check, two
independent consumers of one spec.
2. **Operator designation: future decompiler candidate.** `r2dec`
(P-code→SSA→structured C) is the natural engine for the OGAR-Machine's
"semantic shims erode the emulator" direction — recognizing and lifting
stable call surfaces needs decompilation-grade structure recovery, and
building that from scratch was never on any plan. Not scheduled; named.
3. **A second symbolic-execution precedent** (`r2sym`, Z3) in Rust, next to
Ghidra's `SymbolicSummaryZ3` — the branch-population direction now has
two prior arts to study before designing anything.
4. **Access boundary, recorded:** the clone is anonymous/read-only —
pushing or PRs against r2sleigh require re-attaching with push access.
Any fix we need upstream goes through the operator first.

## The integration shape — two roles, both offline

```
Expand Down
36 changes: 34 additions & 2 deletions bench/RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
> | file | what it is |
> |---|---|
> | `results/jmh-run-full.txt` / `jmh-results-full.csv` | the first full sweep (A, B, C/D, and E at 65,536 rows only) |
> | `results/jmh-run.txt` / `jmh-results.csv` | the `./run.sh E_` re-run, after a 256-row arm was added to the fusion sweep |
> | `results/jmh-run.txt` / `jmh-results.csv` | the latest selective run — currently `./run.sh F_` (the W4 row-store facet scan); the earlier `E_` re-run's rows live on in the merged file |
> | `results/jmh-results-merged.csv` | A/B/C from the full sweep + E from the re-run — **the input to every table below** |
> | `results/TABLES.md` | `./summarise.sh results/jmh-results-merged.csv` |
>
Expand Down Expand Up @@ -192,7 +192,7 @@ most of the residual gap. Both are ABI-implementation changes, not ABI-contract

---

## E/F — fusion, and what the fluent API itself costs
## E — fusion, and what the fluent API itself costs

| rows | predicates | `fused` | `unfused` | `fusedScalarKernel` | `planConstructionOnly` | unfused ÷ fused |
|---:|---:|---:|---:|---:|---:|---:|
Expand Down Expand Up @@ -231,6 +231,38 @@ just a list of predicates" should look like.

---

## F — the row-store facet scan (the W4 question)

The question C could not answer: does "the Vector API beats the crossing" survive when the
workload is the REAL substrate layout — 512-byte rows, 32 facets, a *strided* scan doing 32
classid compares per row instead of one compare per 4-byte element? Three arms over the same
`RowStore` (`classId == 9`), every arm's full per-row bitset output cross-checked equal in
`@Setup` before anything was timed (`RowStoreData`'s constructor, at every row count):

| rows | bytes traversed | `native_facetMatch` | `java_vectorApi` | `java_scalar` | native/vector |
|---:|---:|---:|---:|---:|---:|
| 4,096 | 2 MiB | 191.3 ±24.3 µs | **76.1 ±5.3 µs** | 134.7 ±13.9 µs | 2.51× |
| 65,536 | 32 MiB | 3,219.6 ±141.5 µs | **1,674.1 ±126.9 µs** | 3,354.4 ±118.4 µs | 1.92× |
| 1,048,576 | 512 MiB | 79,016.9 ±9,239.4 µs | **69,613.1 ±1,624.1 µs** | 82,541.1 ±3,106.4 µs | 1.14× |

Three findings, in decreasing order of confidence:

1. **The direction survives; the margin collapses.** The Vector API still wins at every measured
row count — but by 1.1–2.5×, not Component C's 56×. More work per byte (32 compares per
512-byte row, the same four-facets-per-512-bit-register algorithm on both sides) is exactly
the regime the § Verdict predicted would narrow the boundary, and now it is measured rather
than predicted.
2. **At 512 MiB everything converges on memory bandwidth.** 69–82 ms to traverse 512 MiB is
~6–7 GB/s on this shared 4-vCPU container — all three arms are bandwidth-bound and the
implementation difference shrinks toward noise (native's CI at 1M rows, ±9.2 ms, brackets much of the
gap to vector).
3. **The native arm carries a disclosed allocation asymmetry.** `RowStore.facetMatches()`
allocates a fresh output segment + view per call, where `NativePattern`'s fused-plan path
reuses a cached scratch mask and the Java arms reuse a `@Setup`-allocated array. At 4,096
rows that fixed cost is a visible slice of 191 µs; at 1M rows it is noise. A
`facetMatchesInto(classId, …)` reuse form is the named follow-up if the small-row gap ever
matters — filed, not assumed to.

## Verdict — where does execution belong?

On the evidence, **not where the architecture currently puts it, for count-only queries.** Stated
Expand Down
10 changes: 9 additions & 1 deletion bench/results/TABLES.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
| 1,048,576 | 4096 | 411.333 ±37.244 | 310.405 ±17.660 | 1623.313 ±26.973 | **java_vectorApi** | 1.33x |
| 4,194,304 | 16384 | 1858.686 ±149.400 | 1319.107 ±37.240 | 6602.036 ±100.771 | **java_vectorApi** | 1.41x |

### E/F — fusion and the cost of the fluent API (µs/op)
### E — fusion and the cost of the fluent API (µs/op)

| rows | predicates | `fused` | `unfused` | `fusedScalarKernel` | `planConstructionOnly` | unfused/fused |
|---:|---:|---:|---:|---:|---:|---:|
Expand All @@ -42,3 +42,11 @@
| 65,536 | 4 | 25.591 ±0.582 | 31.790 ±3.337 | 917.387 ±27.768 | 0.261 ±0.012 | **1.24x** |
| 65,536 | 8 | 58.978 ±4.509 | 60.968 ±2.657 | 1825.916 ±125.373 | 0.601 ±0.032 | **1.03x** |

### F — the row-store facet scan (µs/op, mean ± 99.9% CI)

| rows | row KiB | `native_facetMatch` | `java_vectorApi` | `java_scalar` | fastest | native/vector |
|---:|---:|---:|---:|---:|---:|---:|
| 4,096 | 2048 | 191.285 ±24.278 | 76.108 ±5.316 | 134.712 ±13.871 | **java_vectorApi** | 2.51x |
| 65,536 | 32768 | 3219.554 ±141.518 | 1674.064 ±126.854 | 3354.401 ±118.415 | **java_vectorApi** | 1.92x |
| 1,048,576 | 524288 | 79016.873 ±9239.383 | 69613.100 ±1624.077 | 82541.075 ±3106.426 | **java_vectorApi** | 1.14x |

9 changes: 9 additions & 0 deletions bench/results/jmh-results-merged.csv
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,12 @@ com.adaworldapi.lancegraph.bench.E_FusionAndPlanning.unfused,avgt,1,8,2.097204,0
com.adaworldapi.lancegraph.bench.E_FusionAndPlanning.unfused,avgt,1,8,31.789786,3.337085,us/op,4,65536
com.adaworldapi.lancegraph.bench.E_FusionAndPlanning.unfused,avgt,1,8,4.437382,0.204156,us/op,8,256
com.adaworldapi.lancegraph.bench.E_FusionAndPlanning.unfused,avgt,1,8,60.967976,2.656815,us/op,8,65536
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.java_scalar,avgt,1,8,134.711904,13.870504,us/op,,4096
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.java_scalar,avgt,1,8,3354.401240,118.415079,us/op,,65536
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.java_scalar,avgt,1,8,82541.075065,3106.426103,us/op,,1048576
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.java_vectorApi,avgt,1,8,76.107543,5.315558,us/op,,4096
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.java_vectorApi,avgt,1,8,1674.064233,126.854113,us/op,,65536
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.java_vectorApi,avgt,1,8,69613.100438,1624.076656,us/op,,1048576
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.native_facetMatch,avgt,1,8,191.285057,24.277993,us/op,,4096
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.native_facetMatch,avgt,1,8,3219.553540,141.518033,us/op,,65536
com.adaworldapi.lancegraph.bench.F_RowStoreFacetScan.native_facetMatch,avgt,1,8,79016.872595,9239.382837,us/op,,1048576
Loading