You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Consider an asof-join query T1 ASOF JOIN T2 MATCH_CONDITION(T1.ts1 >= T2.ts2), where ts1 and ts2 are timestamp columns in T1 and T2 with LATENESS values L1 and L2 respectively.
Let TS1 and TS2 be the latest observed timestamps in T1 and T2.
Let WL = min(TS1 - L1, TS2 - L2), i.e., the timestamp before which neither of the two streams can see any updates.
We can GC T1 using integrate_trace_retain_values and the waterline value equal to WL.
Consider an asof-join query
T1 ASOF JOIN T2 MATCH_CONDITION(T1.ts1 >= T2.ts2), wherets1andts2are timestamp columns inT1andT2with LATENESS valuesL1andL2respectively.Let
TS1andTS2be the latest observed timestamps inT1andT2.Let
WL = min(TS1 - L1, TS2 - L2), i.e., the timestamp before which neither of the two streams can see any updates.We can GC
T1usingintegrate_trace_retain_valuesand the waterline value equal toWL.To GC the right side, we need the mechanism described in RFC: Generalizing the GC API to support group operators #1975. Specifically, we need to preserve the latest value preceding
WLin each group and all newer values.