Skip to content

VarBinArray.DictMode.dictReadOff: U8/U16 offset ptypes fall through to 8-byte reads — IOOBE on dict-over-fsst strings #215

Description

@dfa1

Found by round 2 of the Raincloud conformance triage (#205) on uci-magic-gamma-telescope (written by vortex-data==0.69.0). The class (utf8) column is a dict over an FSST values child whose codes_offsets is vortex.sequence(u8, len=3):

root: vortex.dict(utf8?, len=19020)
  codes: vortex.runend(u8, len=19020)
  values: vortex.fsst(utf8?, len=2)
    codes_offsets: vortex.sequence(u8, len=3)   # base 0, multiplier 2

VarBinArray$DictMode.dictReadOff (reader/.../VarBinArray.java:330-335) special-cases only I32/U32 as 4-byte reads and falls through to getAtIndex(LE_LONG, i) for every other offsets ptype. Here the offsets ptype is U8 while the materialized buffer holds 3×4-byte values (12 B), so reading offset index 1 attempts an 8-byte load at byte 8 of a 12-byte segment:

IndexOutOfBoundsException: Out of bound access on segment ...; new offset = 8; new length = 8
    at VarBinArray$DictMode.dictReadOff(VarBinArray.java:333)
    at VarBinArray$DictMode.getBytes(VarBinArray.java:283)

Fix directions (either/both): widen dictReadOff to handle U8/U16 (and I8/I16?) offset ptypes, and/or normalize the offsets ptype where the sequence/FSST decode path materializes offsets into a wider buffer — the ptype metadata and the actual buffer stride must agree.

Repro: hydrate the slug via Raincloud, vortex-cli export uci-magic-gamma-telescope.vortex -.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions