fix(resolver): resolver conditional send and dedup key#20121
fix(resolver): resolver conditional send and dedup key#20121
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- In
TestNonMergeableItemsReplaceOnDuplicate, both pushedtestIteminstances usevalue: 1, so the assertion that the item is "replaced, not merged" isn’t actually verifiable; consider using different values (e.g., 1 then 2) and asserting that the second value wins to make the behavior observable. - The
Mergeable[K comparable]interface doesn’t use its type parameterKin the method signature; you could simplify it totype Mergeable[K comparable] interface{ MergeFrom(old Item[K]) }→type Mergeable[K comparable] interface{ MergeFrom(old Item[K]) }without the type parameter onMergeableitself, or drop the generic onMergeableentirely and just useMergeFrom(old Item[K])with the queue’sK. - In tests like
TestMergeablePreservesQueueOrder, you’re asserting the pulled items with unchecked type assertions (e.g.first := q.PullBlocking(&stopSignal).(*mergeableItem)); consider mirroring the safer pattern used elsewhere (item, ok := ...; require.True(ok)) to avoid panics and make failures easier to diagnose.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `TestNonMergeableItemsReplaceOnDuplicate`, both pushed `testItem` instances use `value: 1`, so the assertion that the item is "replaced, not merged" isn’t actually verifiable; consider using different values (e.g., 1 then 2) and asserting that the second value wins to make the behavior observable.
- The `Mergeable[K comparable]` interface doesn’t use its type parameter `K` in the method signature; you could simplify it to `type Mergeable[K comparable] interface{ MergeFrom(old Item[K]) }` → `type Mergeable[K comparable] interface{ MergeFrom(old Item[K]) }` without the type parameter on `Mergeable` itself, or drop the generic on `Mergeable` entirely and just use `MergeFrom(old Item[K])` with the queue’s `K`.
- In tests like `TestMergeablePreservesQueueOrder`, you’re asserting the pulled items with unchecked type assertions (e.g. `first := q.PullBlocking(&stopSignal).(*mergeableItem)`); consider mirroring the safer pattern used elsewhere (`item, ok := ...; require.True(ok)`) to avoid panics and make failures easier to diagnose.
## Individual Comments
### Comment 1
<location path="pkg/dedupingqueue/deduping_queue_test.go" line_range="104-113" />
<code_context>
+ s.Assert().True(merged.flag2, "flag2 should accumulate across three pushes")
+}
+
+func (s *uniQueueSuite) TestNonMergeableItemsReplaceOnDuplicate() {
+ q := NewDedupingQueue[string]()
+ q.Push(&testItem{value: 1})
+ q.Push(&testItem{value: 1})
+
+ s.Assert().Equal(1, q.queue.Len(), "duplicate key should result in one item")
+
+ stopSignal := concurrency.NewSignal()
+ defer stopSignal.Signal()
+ item := q.PullBlocking(&stopSignal)
+ pulled, ok := item.(*testItem)
+ s.Require().True(ok)
+ s.Assert().Equal(1, pulled.value, "non-mergeable item should be replaced, not merged")
+}
+
</code_context>
<issue_to_address>
**issue (testing):** The non-mergeable replacement test does not actually prove replacement vs keeping the existing item
Because both pushed `testItem`s use the same `value`, and `GetDedupeKey` is derived from `value`, this test still passes if the queue keeps the old item instead of replacing it. To verify "replace, not merge" behavior, decouple the dedupe key from the payload (e.g., use a `key int` for `GetDedupeKey` and a separate `value int` that differs between pushes), and then assert that the pulled item has the `value` from the second push. That way the test fails if the queue ever retains the old instance.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
📝 WalkthroughWalkthroughIntroduces a new Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant DedupingQueue
participant Mergeable Item
participant Resolver
participant OutputQueue
Client->>DedupingQueue: Push(item1 with key="A")
DedupingQueue->>DedupingQueue: Store item1 with key="A"
Client->>DedupingQueue: Push(item2 with key="A", flags differ)
DedupingQueue->>DedupingQueue: Found existing item1 for key="A"
DedupingQueue->>Mergeable Item: Implements Mergeable?
Mergeable Item-->>DedupingQueue: Yes
DedupingQueue->>Mergeable Item: MergeFrom(item1)
Mergeable Item->>Mergeable Item: Merge flags (OR for forceDetection, AND for skipResolving)
DedupingQueue->>DedupingQueue: Remove item1, insert merged item2
Resolver->>DedupingQueue: Pop next item
DedupingQueue-->>Resolver: Merged item with combined flags
Resolver->>Resolver: Resolve with merged flags
Resolver->>OutputQueue: Send ResourceEvent
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@sensor/kubernetes/eventpipeline/resolver/resolver_race_test.go`:
- Around line 600-603: The test currently pre-signals the stop signal
(stop.Signal()) before calling
env.resolver.deploymentRefQueue.PullBlocking(&stop), which can mask a non-empty
queue; remove the pre-signaling so you create an unsignaled stop via
concurrency.NewSignal() and pass it directly to
deploymentRefQueue.PullBlocking(&stop) and assert the returned item is nil;
ensure the concurrency signal is imported/available and that no other code
pre-signals stop before the assertion (refer to concurrency.NewSignal,
stop.Signal/stop.Done, and env.resolver.deploymentRefQueue.PullBlocking).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Central YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: e213fb81-2d50-48d3-a4ab-b2c5c91a2140
📒 Files selected for processing (4)
pkg/dedupingqueue/deduping_queue.gopkg/dedupingqueue/deduping_queue_test.gosensor/kubernetes/eventpipeline/resolver/resolver_impl.gosensor/kubernetes/eventpipeline/resolver/resolver_race_test.go
🚀 Build Images ReadyImages are ready for commit 6a6a8c0. To use with deploy scripts: export MAIN_IMAGE_TAG=4.11.x-711-g6a6a8c0b19 |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #20121 +/- ##
==========================================
+ Coverage 49.70% 49.77% +0.07%
==========================================
Files 2766 2767 +1
Lines 209352 209557 +205
==========================================
+ Hits 104051 104301 +250
+ Misses 97611 97561 -50
- Partials 7690 7695 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sourcery-ai you can't reference K in MergeFrom(old Item[K]) without K being a type parameter on the interface |
Description
DO NOT REVIEW (until I address the AI reviews)
The
ROX_AGGREGATE_DEPLOYMENT_REFERENCE_OPTIMIZATIONfeature flag was disabled in May 2024 (PR #11097) because it caused SummaryTests e2e flakes. The optimization uses aDedupingQueueto deduplicate deployment references, reducing the O(secrets × deployments) amplification that causes OOM in large clusters. This PR fixes two correctness issues that likely caused those flakes.Problem 1 — conditional send drops ReprocessDeployments data. When
resolveDeploymentreturns false (deployment not yet built, not found, or build error),runPullAndResolveskips sending the event. But ifforceDetectionwas true,ReprocessDeploymentsdata (deduper reset) was already added to the event and gets silently dropped. In the FF-disabled path,processMessagesends unconditionally, so the data is preserved. The fix sends the event wheneverReprocessDeploymentshas data, regardless of the resolve result.Problem 2 — dedup key includes flags, preventing cross-type deduplication. The old key
{id}-{action}-{skipResolving}-{forceDetection}means a CentralUpdatedImageref and a K8s deployment update ref for the same deployment occupy separate queue slots. They're resolved independently, and the skip-resolve path can hit theisBuilt=falseguard when the deployment is mid-update. The fix simplifies the key to{id}-{action}and merges flags on duplicate push (forceDetectionsticky-true,skipResolvingsticky-false). A merged ref takes the full resolve path with forced detection, bypassing theisBuiltguard entirely.isBuilt=false, no K8s ref in queueisBuilt=falseforceDetectionref but deployment not found or build errorUser-facing documentation
Testing and quality
Automated testing
How I validated my change