Commit b0aea59
committed
simd: a codegen oracle, and the u64 rotate it immediately found
Adds `crates/simd-codegen-oracle` (+ script, baseline, CI job) and removes
blake3's C/asm FFI. The oracle answers "does this need intrinsics?" with an
assembly measurement instead of intuition — and its first run falsified the
design assumptions it was built to check.
## Why
TD-T22 measured that under the pinned `-Ctarget-cpu=x86-64-v3`, the
"scalar polyfill" SIMD types already compile to packed AVX2 at the
instruction floor. A PR had hand-written 700 lines of intrinsics against
that nonexistent gap. Nothing in the repo could have caught it: there is no
codegen check, only correctness parity harnesses.
## What the oracle measured
13 probes, three groups. Group A expected to vectorize, Group B expected
NOT to, Group C (u64 rotate) deliberately unclassified.
Group A — all vectorized, as expected. `arx_rounds_u32x16` (10-round ChaCha
double-round): 52 packed, 0 scalar arithmetic on lane data.
Group B — **3 of 5 predictions were WRONG.** LLVM vectorized:
* saturating_abs_i8x32 → vpxor/vpsubsb/vpblendvb, i.e. it synthesized
the VPABSB abs+clamp trick on its own
* widening_u16_to_f32 → vpmovzxwd + vcvtdq2ps
* cross_lane_reverse_u8x64 → vbroadcasti128 + vpshufb + vpermq — a
cross-lane permute, from a scalar index loop
Only serial_dependent_chain (loop-carried dependency) and gather_lookup_u8
stayed scalar. The "cross-lane/widening/saturating obviously needs
intrinsics" intuition is measurably false.
Group C — **the u64 rotate does NOT vectorize.** rot_u64x8 / rot_u64x4:
0 packed, one scalar `rorq %cl` per lane. blake2b_g_u64x8: the leading
`a+b` goes packed (vpaddq), then LLVM extracts every lane to a GPR and
stays scalar through all four rotate stages — including the byte-granular
amounts 32/24/16, which at u32 width fold to vpshufb. AVX2 has
vpsllq/vpsrlq and LLVM applies exactly that shift-or to u32 rotate-by-12/7;
it declines to at 64-bit width.
That matters because BLAKE2b is a 64-bit ARX cipher and argon2 uses
BLAKE2b. The crate has 8 `rotate_left` methods (all u32) and zero
`rotate_right` at any width, so argon2's kernel is unexpressible today.
This is the first intrinsic override meeting the entry criterion: a probe
proving the generic form fails.
Same crate, same week, opposite answers for u32 and u64. That contrast is
the argument for the oracle.
## blake3: no more C
Operator directive — C is a contamination of ndarray, and vendored builds
fail without a C toolchain. blake3's default build ran cc over
c/blake3_{sse2,sse41,avx2,avx512}_x86-64_unix.S: measured 33 .o files plus
libblake3_avx512_assembly.a, with blake3_*_ffi cfgs set. Now pinned to
`default-features = false, features = ["pure"]`, which routes build.rs to
build_sse2_sse41_avx2_rust_intrinsics() ("No C code to compile here").
Verified after `cargo clean -p blake3`: the only cfgs actually SET are
blake3_{sse2,sse41,avx2}_rust; 0 object files, 0 archives, no cc. (The
_ffi names still appear in the build output as rustc-check-cfg
DECLARATIONS — cargo listing valid cfg names — not assignments. A grep
that cannot tell the two apart reports a false positive here.)
Remaining cc build-deps in the lockfile (cmake, openblas-build,
openssl-sys) are reachable only through the optional `blas` feature, not
`default = ["std", "hpc-extras"]`. blake3 was the only C on the default
path — which is why vendoring broke for everyone.
## Honesty rule baked into the tool
The analyzer separates "scalar arithmetic on lane data" from loop control,
so a trip-counter `decl` is never reported as scalar lane work. That exact
overclaim was made and corrected earlier this week; the tool cannot
reproduce it.
## Docs
* .claude/knowledge/simd-one-spec-design.md — design for collapsing 5
backends (31 macro-generated + 57 hand-written types, 13,253 LoC, three
authoring strategies) into one spec, with the oracle as the entry
criterion for intrinsic overrides. Includes the table of my three wrong
predictions.
* .claude/knowledge/crypto-lane-status.md — u32 ARX proven optimal
(ChaCha20/BLAKE3 need no SIMD work); u64 ARX absent and now measured as
genuinely needing intrinsics.1 parent 2c8af9a commit b0aea59
10 files changed
Lines changed: 1533 additions & 2 deletions
File tree
- .claude/knowledge
- .github/workflows
- crates/simd-codegen-oracle
- baselines
- src
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
166 | 182 | | |
167 | 183 | | |
168 | 184 | | |
| |||
388 | 404 | | |
389 | 405 | | |
390 | 406 | | |
| 407 | + | |
391 | 408 | | |
392 | 409 | | |
393 | 410 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
185 | 185 | | |
186 | 186 | | |
187 | 187 | | |
188 | | - | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
189 | 214 | | |
190 | 215 | | |
191 | 216 | | |
| |||
402 | 427 | | |
403 | 428 | | |
404 | 429 | | |
405 | | - | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
406 | 437 | | |
407 | 438 | | |
408 | 439 | | |
| |||
0 commit comments