Skip to content

fix: 🐛 Avoid duplicate profile capture resume scans - #83

Merged
oxspan merged 1 commit into
developfrom
feature/74
Nov 12, 2026
Merged

oxspan merged 1 commit into
developfrom
feature/74

Conversation

@oxspan

@oxspan oxspan commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Problem

Each incomplete profile-capture resume validated every committed page, then read and decoded the same pages again to collect transaction IDs. Repeated 40-page invocations increased local preparation work without downloading those committed profiles again.

Refs #74

Changes

  • Collect IDs through the existing validated record traversal and remove the second full-prefix read loop. Keep checksum, record/scope, profile and global duplicate validation before recovery or append.
  • Preserve unavailable-profile recovery and already-complete resumes, incoming duplicate rejection, capture formats and CLI budgets.
  • Add a real-file JFR regression for read bytes and an incoming-duplicate regression. Neither requires a production instrumentation hook or timing threshold.
  • Add opt-in resumeReadCheck with reproducible counts, bytes, scan histograms, preparation/capture timing and implementation class fingerprints. The existing -PcliJar=... Gradle property selects a baseline JAR for comparison; no CLI flags are added.
  • Document residual scaling, memory tradeoffs and the contract needed for bounded same-process continuation.

Validation

  • Full Linux tests, coverage and release-input checks passed: 251 XML entries, zero failures/errors/skips; one entry represents an empty Windows-only specification. Line coverage 95.35%, branch coverage 80.51%, against unchanged 90%/80% floors.
  • Thirteen packaged CLI cases passed: partial/resumed/completed capture, unchanged completed bytes, corruption rejected before append, unavailable-profile recovery in partial and completed captures, inspect and adjacent v1 capture.
  • The baseline fails the new I/O regression by reading twice each committed page's bytes. Removing ID collection from the new traversal fails the incoming-duplicate test; restoring it passes the full suite.

Synthetic measurement

Linux/WSL2, JDK 21, 512 MiB maximum heap; three fresh JVM trials per version, real EvidenceStore, synthetic profiles and no Collector/network delay. Each workload captures 120 pages in three 40-page application invocations, then resumes the completed capture. Workloads use 2 or 100 records/page with 20 small structural SQL steps/profile. All generated page fingerprints match between versions and trials.

Ordinary existing-prefix scans change from 0, 80, 160 to 0, 40, 80. The two incomplete resumes read 240 → 120 existing pages. The completed-resume control still validates all 120 pages once in both versions; committed profiles are not fetched again.

Medians for the 100-record/page workload:

Existing pages / records Existing page bytes Read bytes before → after Preparation ms before → after Whole invocation ms before → after
40 / 4,000 10,561,786 21,123,572 → 10,561,786 177.373 → 90.160 273.139 → 165.645
80 / 8,000 21,125,786 42,251,572 → 21,125,786 305.957 → 162.195 374.769 → 231.992

Run ./gradlew --no-daemon resumeReadCheck to generate a new report and raw JFR recordings. See measurement instructions for baseline comparison and metric definitions.

This halves ordinary prefix scans while cumulative work across separate invocations remains quadratic. Recovery can reread unavailable-profile pages. The application ID set and store uniqueness set are now live together during validation; peak heap was not measured. Timings exclude JVM startup and recorder start/dump/stop, retain event overhead, and do not control cache, JIT, GC, contention or candidate order. They do not establish production throughput, native Windows/macOS performance or a universal latency improvement.

@github-actions

Copy link
Copy Markdown
Contributor

Linux CI: success · Commit 3e1380a

Test results (Linux)

Total Passed Failed Errors Skipped
251 251 0 0 0

Test coverage

Metric Covered / total Coverage Minimum Result
LINE 2192 / 2299 95.35% 90.00% PASS
BRANCH 1289 / 1601 80.51% 80.00% PASS

Includes the test JVM and CLI subprocesses. Excludes optional real-Collector smoke tests.

Workflow logs and report artifacts

Updated on each CI run. Windows checks, when required, are reported separately in the workflow.

@oxspan
oxspan merged commit defecb2 into develop Nov 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant