Flat variant serialization bench - #6843
Merged
Merged
Conversation
Generate 100,000 documents of exactly 16 KiB and serialize all of them
with DbspSerializer, once through to_bytes, which allocates an FBuf per
document, and once into a single recycled FBuf, the way the layer-file
writer packs a data block. The gap between the two modes is buffer
allocation, teardown, and first touch, so the shared-buffer rate is the
ceiling the encoding can reach.
cargo bench -p feldera-sqllib --bench flat_variant_serialize
cargo bench -p feldera-sqllib --bench flat_variant_serialize -- \
--docs 10000 --size 4096
An unoptimized build runs 200 documents instead of 100,000, because
cargo test --benches runs a harnessless bench binary with no arguments
and its rates would mean nothing anyway.
Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ArchivedVec::serialize_from_slice resolves a slice element by element
unless rkyv's nightly-only copy feature is on, so serializing a document
cost one write call per byte rather than the single bulk write the module
documentation claims. serialize_copy_from_slice writes it once. A byte is
trivially copyable, has no padding, and archives as itself, so the copy
is exactly the encoding.
Measured by benches/flat_variant_serialize.rs on 100,000 documents of
16 KiB, paired back to back in one machine state:
mode before after
fresh FBuf per document 3.689 s 424 MiB/s 0.056 s 27 GiB/s
shared FBuf 3.574 s 438 MiB/s 0.033 s 47 GiB/s
Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
The derived rkyv::Serialize delegated to SmallVec's, which resolves the
payload element by element, one write call per byte. Hand-writing the impl
around serialize_copy_from_slice writes it once. A byte is trivially
copyable, has no padding, and archives as itself, so the copy is exactly
the payload, and the archived layout does not change: an ArchivedVec<u8>
either way, so existing storage stays readable.
Every BINARY and VARBINARY value pays this on each batch write to storage
and each checkpoint, and so does Variant::Binary. Measured on 16 KiB
values, against the untouched Vec<u8> path as a control in the same run:
Vec<u8> (per element) 97.808 ms 0.31 GiB/s
ByteArray (bulk copy) 1.118 ms 27.30 GiB/s
Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
blp
approved these changes
Aug 11, 2026
blp
left a comment
Member
There was a problem hiding this comment.
Do we ever serialize FBuf? It could also use such an optimization, I think, in crates/storage/src/fbuf.rs.
Comment on lines
+60
to
+63
| // rkyv::Serialize is hand written rather than derived so that the payload is | ||
| // copied in one bulk write. The derived impl delegates to SmallVec's, which | ||
| // resolves the bytes element by element, one write call per byte, and measured | ||
| // 47x slower on a 16 KiB value. |
Comment on lines
+1498
to
+1500
| /// A bulk copy of an offset table is only valid where a `usize` archives as | ||
| /// itself. rkyv archives integers in native byte order, and the `size_64` | ||
| /// feature makes the archived width 8, so this holds on every 64-bit target. |
Member
There was a problem hiding this comment.
Should we assert that the platform is little-endian?
Contributor
Author
There was a problem hiding this comment.
I don't think the endianness behavior has changed: everything is serialized in host order.
gz
approved these changes
Aug 11, 2026
The three batch serializers end by writing a Vec<usize> of offsets, one
entry per key, value and weight. Vec's own rkyv impl resolves it element
by element, one write call per offset. serialize_offsets wraps the table
in rkyv's CopyOptimize so it lands in one write instead.
The wrapper leaves the archived layout alone, an ArchivedVec of the same
elements, so the readers still take archived_root::<Vec<usize>> and
existing checkpoints stay readable. Copying is only valid where a usize
archives as itself, which a static assertion pins: rkyv archives integers
in native byte order, and the size_64 feature fixes the archived width at
8 bytes, so it holds on every 64-bit target.
The table itself serializes 25x faster (3M offsets: 181.6 ms to 7.3 ms).
End to end the gain is smaller, because the per-item serialization of the
keys, values and weights dominates and is untouched. Checkpointing a
1M-row VecIndexedWSet, paired back to back:
before 35.264 ms/pass 1.48 GiB/s
after 30.874 ms/pass 1.69 GiB/s
Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
Checkpoint::checkpoint returns state already serialized, and the committed
structs that carry it hold it as Vec<u8>, whose rkyv impl copies it back
out one byte per write call. rkyv's CopyOptimize wrapper writes it once.
Annotated: CommittedZ1 and CommittedTransactionZ1, which carry a whole
operator state, and CommittedClock, which is small but free to fix.
CommittedWindow keeps the per-element path, because the wrapper applies to
a Vec field rather than through an Option of a tuple and a pair of
serialized keys is not worth reshaping the struct for; the reason is now
recorded there.
Serializing a CommittedZ1 with a 16 MiB blob, paired back to back:
before 13.304 ms/pass 1.17 GiB/s
after 1.419 ms/pass 11.01 GiB/s
Restore gets faster too, since the wrapper deserializes with one
copy_nonoverlapping instead of a per-element loop.
The archived layout is unchanged, so checkpoints written by earlier builds
still restore. bulk_copy_keeps_the_committed_layout pins exactly that: it
compares the archived bytes against a reference struct that has no wrapper,
and fails if the wrapper ever changes the layout, which a swap to AsBox
confirms.
Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ByteArray::size_of_children delegated to its SmallVec field, and there is no SizeOf impl for SmallVec, so the call resolved to the one for [u8]: it walked the bytes, reported none of the buffer holding them, and left a 16 KiB VARBINARY value claiming 48 bytes. Count the spilled allocation the way Vec counts its own. An inline payload needs nothing added, being part of size_of::<ByteArray>() already. The undercount was total rather than approximate, and BINARY and VARBINARY are the only SQL types affected: SqlString wraps an ArcStr, which feldera-size-of does support, and FlatVariant reports its Arc buffer. What read the wrong number: the fallback batches size their contents with size_of().total_bytes() per key, value and weight (trace/ord/fallback/indexed_wset.rs:568-654) to decide when a batch moves to storage, so a batch of large binary values looked nearly free and stayed in memory. Expect such batches to spill sooner now, which is the point. Dropping the delegation also drops a per-byte loop that computed nothing. Found by the FlatVariant serialization benchmark, whose VARBINARY corpus reported 4.6 MiB of heap for 1.5 GiB of payloads. Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
ryzhyk
force-pushed
the
flat-variant-serialization-bench
branch
from
August 12, 2026 00:56
57d96a9 to
0102a23
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Several similar fixes to rkyv serialization that use efficient memcopy instead of element-by-element serialization for types whose rkyv's representation is identical to their in-memory layout.
Describe Manual Test Plan
Checklist
Breaking Changes?
Mark if you think the answer is yes for any of these components:
Describe Incompatible Changes