Skip to content

[SQL] GC for JoinFilterMap operator was applied on the wrong side - #6830

Merged
mihaibudiu merged 1 commit into
feldera:mainfrom
mihaibudiu:issue6829
Aug 9, 2026
Merged

[SQL] GC for JoinFilterMap operator was applied on the wrong side#6830
mihaibudiu merged 1 commit into
feldera:mainfrom
mihaibudiu:issue6829

Conversation

@mihaibudiu

Copy link
Copy Markdown
Contributor

Fixes #6829

Checklist

  • Unit tests added/updated

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
@mihaibudiu
mihaibudiu requested a review from ryzhyk August 7, 2026 23:24
final DBSPExpression func = new DBSPTupleExpression(monotoneFields, false);
final OutputPort extractLeft = this.createApply(leftLimiter, join, func.closure(var));

if (INSERT_RETAIN_VALUES) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was the bug: notice how the GC inferred from the left is applied to the left input
there's a similar mistake for the right input

@mythical-fred mythical-fred left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness fix looks right. The prior code used each side's own delta-path bound to GC that same side's input trace; that only works when the two paths carry the same monotone column, which is not the case for a JoinFilterMap whose inequality relates two different timestamp columns. Trace on side X can only be pruned once we know values on side X can never join a future arrival on side Y, so the retention closure has to be extracted from the delta bound of side Y. Swapping the two if (leftLimiter/rightLimiter) blocks accomplishes exactly that.

The extraction of joinInputRetention(parameterNumber, oppositePathTuple, iomap, valueType, keyPart) replaces the manual varIndex bookkeeping with oppositePathTuple.compressedIndex(outputField). That is cleaner and less error-prone; the two prior copies of that loop had subtly different structures (the right-hand branch pre-walked left fields to advance varIndex), and both are now the same code path.

Tests are excellent: issue6829, issue6829Windowed, issue6829Asof each pair the LATENESS variant against a lateness-free control that must produce identical outputs but must also have zero GC operators. The trace-diagram comments walking through the bound at each step will pay for themselves the next time someone edits this file. checkJoinGC asserts both the presence of the expected retainers and the absence of any others.

APPROVE.

@mihaibudiu
mihaibudiu added this pull request to the merge queue Aug 8, 2026
@mihaibudiu

Copy link
Copy Markdown
Contributor Author

I am merging this, since it's a clear bug fix

Merged via the queue into feldera:main with commit 3873028 Aug 9, 2026
1 check passed
@mihaibudiu
mihaibudiu deleted the issue6829 branch August 9, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LATENESS violation in as-of-like inequality join + arg_max

3 participants