Skip to content

Enforce continuation diagnostics in suite fixtures - #12528

Draft
amarziali wants to merge 27 commits into
andrea.marziali/diag2from
andrea.marziali/diagnose-suite-fixture-continuations
Draft

amarziali wants to merge 27 commits into
andrea.marziali/diag2from
andrea.marziali/diagnose-suite-fixture-continuations

Conversation

@amarziali

Copy link
Copy Markdown
Contributor

What Does This Do

Extends the default continuation-leak diagnostics to suite-level test fixtures:

  • Spock setupSpec() and cleanupSpec()
  • JUnit @BeforeAll and @AfterAll

This closes the diagnostic window around work created before the first test or cleaned up after the last test. Reports identify whether a failure belongs to suite setup, an individual test, or suite cleanup.

Class-level @TrackScopeContinuations configuration applies to suite fixtures. The investigation skill has also been updated to describe the expanded coverage.

flowchart LR
    A["Harness initializes tracer"] --> B["Suite setup<br/>setupSpec / @BeforeAll"]
    B --> C["Per-test diagnostics"]
    C --> D["Suite cleanup<br/>cleanupSpec / @AfterAll"]
    D --> E["Harness closes tracer"]

    B -. "report suite setup leaks" .-> F["Diagnostic report"]
    C -. "report test leaks" .-> F
    D -. "report suite cleanup leaks" .-> F
Loading

Motivation

Continuation work created in suite fixtures previously fell outside the per-test diagnostic window. This could hide lifecycle problems caused during shared fixture initialization or shutdown.

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@datadog-prod-us1-5

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.01 s 13.99 s [-0.6%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent 12.99 s 12.97 s [-0.5%; +0.9%] (no difference)
startup:petclinic:appsec:Agent 17.03 s 16.96 s [-0.5%; +1.3%] (no difference)
startup:petclinic:iast:Agent 17.11 s 17.05 s [-0.5%; +1.2%] (no difference)
startup:petclinic:profiling:Agent 16.76 s 16.92 s [-2.1%; +0.2%] (no difference)
startup:petclinic:sca:Agent 16.94 s 16.85 s [-0.5%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 16.19 s 16.09 s [-0.4%; +1.7%] (no difference)

Commit: 8314d045 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter

pr-commenter Bot commented Sep 16, 2026

Copy link
Copy Markdown

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch andrea.marziali/diag2 andrea.marziali/diagnose-suite-fixture-continuations
git_commit_date 1789558899 1789650722
git_commit_sha f6f5508 8314d04
See matching parameters
Baseline Candidate
ci_job_date 1789652054 1789652054
ci_job_id 2053394970 2053394970
ci_pipeline_id 138164902 138164902
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-sdztcad8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-sdztcad8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter

pr-commenter Bot commented Sep 16, 2026

Copy link
Copy Markdown

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch andrea.marziali/diag2 andrea.marziali/diagnose-suite-fixture-continuations
git_commit_date 1789558899 1789650722
git_commit_sha f6f5508 8314d04
See matching parameters
Baseline Candidate
ci_job_date 1789652093 1789652093
ci_job_id 2053394971 2053394971
ci_pipeline_id 138164902 138164902
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-2lm8nq30 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-2lm8nq30 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume unsure
[+660.810op/s; +9293.947op/s] or [+0.400%; +5.632%]

@amarziali
amarziali force-pushed the andrea.marziali/diag2 branch from 13947ba to c034c2c Compare September 18, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant