Skip to content

AlpRdEncodingDecoder drops left_parts validity — nulls decode as 0.0 (silent loss) #234

Description

@dfa1

Found by Raincloud triage round 5 (#205) on bi-euro2016, columns polarity_confidence (109047 nulls) and subjectivity_confidence (26201 nulls), both f64?. Same silent-null-loss class as the fixed #210/#225/#226, but in a decoder none of those touched: vortex.alprd (ALP real-double).

ALPRD reconstructs f64 = f(left_parts, right_parts); nullability rides on the nullable left_parts child. The decoder does not propagate that child's validity, so null rows decode to 0.0.

Array tree (chunk 0):

polarity_confidence: vortex.alprd(f64?, len=131072)
  metadata: right_bit_width: 50
  left_parts: fastlanes.bitpacked(u16?, len=131072)
    validity_child: vortex.bool(bool, len=131072) [sum=125297]   # 5775 nulls, dropped
  right_parts: fastlanes.bitpacked(u64, len=131072)

[polarity_confidence row 26] java='0.0' parquet=None (also rows 55, 81, 128, 143).

Contrast: plain vortex.alp over a bitpacked-with-validity_child child decodes nulls correctly (fixed in #210), so this is specific to the vortex.alprd path. Fix: AlpRdEncodingDecoder must lift the left_parts MaskedArray validity and wrap the reconstructed f64 output in a MaskedArray, mirroring AlpEncodingDecoder's #210 fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions