Conversation
This comment has been minimized.
This comment has been minimized.
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Kafka / consumer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
|
Heads up: the That test disables the Kafka integration and asserts no This will need a system-tests-side update before merge: either exempt/adjust |
Kafka / producer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
Implements the pre-instrumentation config accessor for the data streams feature flag, mirroring the existing usmEnabled pattern. Reads the same data.streams.enabled key that Config.isDataStreamsEnabled() uses, enabling DSM to work independently of the general DD_INTEGRATIONS_ENABLED APM gate. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ledOverridesFalse Save and restore the original dataStreamsEnabled value instead of hardcoding false in the finally block. This prevents test pollution where an incorrect state could leak to the next test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add DATA_STREAMS sampling mechanism (byte value 14) to support decoupling Kafka DSM from APM tracing. This mechanism enforces USER_DROP priority for DSM-only spans and allows overriding locked priority when DSM is enabled. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Use qualified PrioritySampling.USER_DROP form in validateWithSamplingPriority to match the DATA_JOBS case style (line 70) which uses qualified form. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…reams base class Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
When Kafka tracing is disabled but Data Streams Monitoring is enabled, produce-path spans created without a propagated trace context should be marked with PrioritySampling.USER_DROP to avoid billing as APM spans. This implements the 'local-root spans only' scoping from the spec: spans with an extracted parent context (genuine distributed traces) are left completely untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eams base class Change extends InstrumenterModule.Tracing to extends InstrumenterModule.DataStreams for all 8 instrumentation classes in the kafka-clients-3.8 module. This decouples Data Streams Monitoring from APM tracing and prepares for billing-suppression logic. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…me paths) Add TRACING_ENABLED constant and billing-suppression logic for produce and consume paths to suppress unnecessary billing when APM tracing is disabled but DSM is enabled. The suppression only applies to local-root spans (no extracted parent context) per the DSM-Kafka decoupling plan. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ensures the cross-language contract holds: when APM tracing is disabled and DSM is enabled, no span is ever written to the agent for Kafka instrumentations. Replaces AgentTracer.startSpan with a lightweight Utils.newPathwayOnlySpan shim across kafka-clients-0.11, kafka-clients-3.8, and kafka-streams-0.11, and extracts shared DSM checkpoint/inject and transaction-tracking logic into helper methods to remove duplication. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… mechanism InstrumenterConfig.java unconditionally disabled dataStreamsEnabled under GraalVM native-image, unlike its pre-PR behavior; DSM has no native-image restriction, so its read is moved outside that carve-out to match dataJobsEnabled. SamplingMechanism.DATA_STREAMS forced a USER_DROP sampling priority to keep DSM-only Kafka spans from being billed as APM traces. That approach is now fully superseded by Utils.newPathwayOnlySpan, which never creates a real span for the DSM-only case, so DATA_STREAMS had zero remaining production callers and is removed as dead code. Also collapses the near-identical TRACING_ENABLED computation duplicated across the three Kafka decorators (kafka-clients-0.11, kafka-clients-3.8, kafka-streams-0.11) into a single Utils.isTracingEnabled helper in kafka-common, per techdebt review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
bb5d6cf to
c498c71
Compare
Several kafka-clients instrumentations referenced KafkaDecorator (whose TRACING_ENABLED field now calls Utils.isTracingEnabled) without declaring kafka_common.Utils as a helper class, causing muzzle to fail resolving that dependency. Separately, kafka-streams' StartSpanAdvice/StartSpanAdvice27 advice classes contained their own private static helper methods, which muzzle cannot validate since advice classes aren't part of the checked helper set. Moved that logic into KafkaStreamsDecorator, which is already a registered helper class. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Claude (re-review update): Checked back in on this since my 08-31 review — the three substantive findings from back then are all resolved via a genuine redesign rather than a patch:
Two other things worth noting for reviewers:
Scope has grown since my last pass to also cover |
BC_IMPOSSIBLE_INSTANCEOF flags the Object-cast instanceof check used to detect RecordMetadata support (only available from kafka-streams 2.7+), which SpotBugs can't reason about statically but is always true at runtime given this class's version constraint. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…-trace-java into worktree-dsm-kafka-decoupling
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
The two most critical issues are: (1) AgentInstaller never installs CoreTracer when tracing is disabled but DSM is enabled, so every DSM checkpoint is silently dropped via the no-op implementation; and (2) the DSM-only span/carrier zeroes out trace and span IDs, severing continuity for any active customer trace and its producer callbacks.
🤖 Datadog Autotest · Commit 820ab35 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 820ab35e6d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Several branch-selection checks used the dynamic, remote-config-aware traceConfig().isDataStreamsEnabled() alongside the static TRACING_ENABLED flag when deciding whether to create a real APM span vs. a DSM-only pathway shim. That let toggling DSM off at runtime silently re-enable real Kafka spans that should stay disabled. Also fixes producer callbacks losing the active trace, and restores strict span-write ordering between broker/queue spans and their consume-span children. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tale span If an exception was thrown between startTracedConsumeSpan setting pendingQueueSpanToFinish and it being finished/nulled in startNewRecordSpan, the field stayed set and a later, unrelated call would finish that stale queue span at the wrong time. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ericfirth is this still exploratory, or is this something you'd like to integrate before the next release? |
@mcculls No, not really exploratory anymore. We'd like to merge it |
Summary
InstrumenterModule.DataStreams, so DSM checkpoints are created even when APM tracing/integrations are disabled.PrioritySampling.USER_DROPviaSamplingMechanism.DATA_STREAMSwhen they are the local root span, so they aren't billed as APM — while never dropping an ambient customer trace.KafkaDecorator's hardcoded integration-name list into shared constants reused byTRACING_ENABLEDand every module constructor, delegatesConfig.isDataStreamsEnabled()toInstrumenterConfig, extracts a duplicated DSM-drop guard inKafkaStreamTaskInstrumentationinto a helper, and rewritesInstrumenterModuleTestto use@WithConfiginstead of raw reflection.Motivation
Using
dd-trace-rbwith JRuby doesn't support DSM. This explores whether Java's DSM instrumentation can run standalone (independent of APM tracing) as a path back to DSM support when tracing is handled by a different tracer, such asdd-trace-rbJira ticket: [DSM2-278]
Test plan
./gradlew :dd-java-agent:instrumentation:kafka-clients-0.11:test./gradlew :dd-java-agent:instrumentation:kafka-clients-3.8:test./gradlew :dd-java-agent:instrumentation:kafka-streams-0.11:test./gradlew :internal-api:test./gradlew :dd-java-agent:agent-tooling:test🤖 Generated with Claude Code