simd: mask/facet primitives for the lance-graph-java membrane (W1a) - #279
Conversation
Slice-level (simd_int_ops.rs, re-exported via ndarray::simd): eq_u32_to_mask, gt_i32_to_mask, eq_u32_strided_to_mask (the AoS-facet classid scan: LE u32 field at first_offset + i*stride, overflow-checked bounds, scalar gathers + U32x16::eq_bitmask 16-wide compare), mask_and/mask_or (+_assign), masked_sum_i32 (walks set bits via trailing_zeros to keep the widened-i64 contract safe from 16-lane i32 accumulation overflow), plus the popcount_batch_u64 re-export. Lane-level: U32x16::eq_bitmask and I32x16::gt_bitmask across all five backends — native mask-register intrinsics on AVX-512 (the only two unsafe blocks, SAFETY-commented), index loops elsewhere per the codegen-oracle rule that intrinsics must be earned. simd_soa: MultiLaneColumn gains the u32 lane (iter_u32x16/len_u32x16) — the lane width a 512-byte facet row store scans classids with: one 64-byte chunk holds four 16-byte facets, classids at U32x16 positions 0/4/8/12, one eq_bitmask answers all four. Gates: simd_int_ops 46/46, simd_soa 15/15, full simd suite green, doctests green, clippy -D warnings clean, fmt clean. Both x86 arms (v3/v4) previously verified byte-identical across the fixed-seed corpus. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs
|
Important Review available on request
Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_5c8da36e-816a-484f-9f9b-2e6cf7fe86c1) |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
The
ndarray::simdprimitives theAdaWorldAPI/lance-graph-javamembrane consumes — previously living only as working-tree state in the shared container (a real container-reset time bomb, now defused by landing them).Slice level (
simd_int_ops.rs, re-exported viandarray::simd)eq_u32_to_mask/gt_i32_to_masku64mask words, LSB-first, trailing bits structurally zero (destination zeroed first)eq_u32_strided_to_masku32field atfirst_offset + i*stride_bytes, overflow-checked bounds (a pathological stride panics rather than wrapping into a bogus in-bounds read), scalar gathers +U32x16::eq_bitmask16-wide compare. At row strides ≥ one cache line the walk is memory-bound, so SIMD earns its keep in the compare, not the loads — stated in the doc rather than impliedmask_and/mask_or(+_assign)masked_sum_i32trailing_zeros— deliberately NOT a 16-lane reduce, which would silently overflowi32accumulation and break the widened-i64contract (reasoned in the doc so nobody "optimizes" it back)Lane level — all five backends
U32x16::eq_bitmask,I32x16::gt_bitmask: native mask-register intrinsics on AVX-512 only (the twounsafeblocks, SAFETY-commented — the compare's result is a__mmask16, no extraction step), index loops on AVX2/NEON/wasm/scalar per the codegen-oracle rule that an intrinsic must be earned. Same shape as the pre-existingcmpge_zero_maskconvention — no new mask type minted.simd_soa:MultiLaneColumngains the u32 laneiter_u32x16/len_u32x16— the lane width a 512-byte facet row store (64K × 32 lanes × (4 B classid + 12 B payload), the lance-graph V3 facet shape) scans classids with: one 64-byte chunk holds four 16-byte facets, classids atU32x16positions 0/4/8/12, oneeq_bitmaskanswers all four. Closes exactly the gap that had madeMultiLaneColumnunusable for that consumer.Gates
simd_int_ops46/46 (incl. 5 new strided tests: independent scalar reference over strides {4, 16, 512, 0}; stride-4 parity vs the contiguous primitive; out-of-bounds and overflowshould_panic; count=0 overwrite),simd_soa15/15, fullsimdfilter 263 passed / 0 failed, doctests green.cargo clippy --lib --tests -- -D warningsclean,cargo fmt --checkclean.-Ctarget-cpu=x86-64-v3andv4) previously verified byte-identical across the fixed-seed corpus; codegen probe confirmed 1×vpcmpeqd/vpcmpltdper group on both arms.W1a contract notes
Slice-tier free functions follow the existing
add_i8/dot_i8file convention; the "struct methods" litmus governs the lane tier, where both new methods live on the typed wrappers. Consumer:lance-graph-java'snative/lgj-abi(itskernels.rsis the sole ndarray-importing file there, all imports viandarray::simd::*).Generated by Claude Code