Skip to content

Require strict trace writes in instrumentation tests - #12525

Draft
amarziali wants to merge 21 commits into
andrea.marziali/diag2from
andrea.marziali/require-strict-trace-writes
Draft

amarziali wants to merge 21 commits into
andrea.marziali/diag2from
andrea.marziali/require-strict-trace-writes

Conversation

@amarziali

Copy link
Copy Markdown
Contributor

What Does This Do

Make strict trace writing mandatory for instrumentation tests.

  • Remove InstrumentationSpecification.useStrictTraceWrites().
  • Remove the JUnit InstrumentationTestConfig.strictTraceWrites(...) option.
  • Configure both test harnesses with strict trace writes unconditionally.
  • Remove the now-unnecessary strictness flag from the span and request-context decorators.
  • Document how unresolved continuation failures should be handled.
  • Update the continuation-leakage skill with the same policy.

Motivation

Disabling strict trace writes allowed incomplete traces and continuation lifecycle problems to be hidden. Instrumentation tests should expose those problems so they can be diagnosed and fixed.

If a continuation problem cannot be fixed immediately, the test should be quarantined with @Flaky and a useful reason or tracked issue.

@TrackScopeContinuations(enabled = false, reason = "...") remains available only for a proven incompatibility with the diagnostic itself. It does not disable strict trace writes.

  flowchart TD
      A[Instrumentation test] --> B[Strict trace writes enabled]
      B --> C{Continuation failure?}

      C -->|No| D[Test passes]
      C -->|Real lifecycle problem| E[Fix ownership or cleanup]
      E --> D
      E -->|Cannot fix immediately| F["@Flaky with reason or issue"]

      C -->|Diagnostic incompatibility| G["Disable tracking narrowly<br/>with documented reason"]
      G --> H[Strict trace writes remain enabled]
Loading

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@amarziali
amarziali added this pull request to stack #12526 September 16, 2026 14:11
@datadog-datadog-prod-us1

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.01 s 13.98 s [-0.8%; +1.2%] (no difference)
startup:insecure-bank:tracing:Agent 12.99 s 13.00 s [-1.0%; +0.9%] (no difference)
startup:petclinic:appsec:Agent 17.65 s 17.43 s [+0.4%; +2.2%] (maybe worse)
startup:petclinic:iast:Agent 16.80 s 17.45 s [-7.9%; +0.5%] (no difference)
startup:petclinic:profiling:Agent 17.53 s 17.24 s [+0.3%; +3.1%] (maybe worse)
startup:petclinic:sca:Agent 17.57 s 17.49 s [-0.3%; +1.3%] (no difference)
startup:petclinic:tracing:Agent 16.54 s 16.60 s [-1.2%; +0.6%] (no difference)

Commit: 90d5cd59 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter

pr-commenter Bot commented Sep 16, 2026

Copy link
Copy Markdown

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch andrea.marziali/diag2 andrea.marziali/require-strict-trace-writes
git_commit_date 1789558899 1789567705
git_commit_sha f6f5508 90d5cd5
See matching parameters
Baseline Candidate
ci_job_date 1789569113 1789569113
ci_job_id 2049389067 2049389067
ci_pipeline_id 137870857 137870857
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-xe1b3plh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-xe1b3plh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant