Skip to content

feat: 🔀 Promote Collector diagnostics and capture improvements - #87

Merged
oxspan merged 17 commits into
masterfrom
develop
Nov 12, 2026
Merged

oxspan merged 17 commits into
masterfrom
develop

Conversation

@oxspan

@oxspan oxspan commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Why

Promote the reviewed develop work so users can inspect Collector data with clearer diagnostics and smaller optional profile output.

Changes

  • Add stored-date object lookup, opt-in dictionary text and live profile timing/network details; show approved Agent setting values.
  • Add optional SQL-hash grouping for live profiles and offline inspection while retaining individual steps as the default and preserving capture formats.
  • Improve command help and option diagnostics, reject reported Collector time-zone mismatches, distinguish invalid capture paths, and briefly retry denied Windows atomic replacements.
  • Collect resume transaction IDs during integrity validation instead of rereading every committed page; document remaining cumulative work and provide a reproducible measurement.
  • Enforce coverage thresholds and report PR test results; update CI action pins and English/Korean guidance.

Validation

This PR's complete hosted Linux and native Windows suites and both platform package checks passed in run 43257298721 for head df26cd3. Line coverage 95.35%, branch 81.08% with unchanged thresholds. The promotion tree and canonical JAR match the corrected candidate verified in run 43251124659. Both platform ZIP checksums and shared JVM payloads were verified; the native Windows command downloaded from this PR passed version/help/error/path/offline-inspection checks with synthetic capture bytes unchanged. GitGuardian checks passed.

Compatibility and risks

Persisted capture formats and strict capture-reader compatibility remain unchanged. Individual profile steps remain the default; live output gains additive diagnostics and step fields, so strict live-output consumers need unknown-key tolerance. --group-sql is optional. Invalid capture paths return actionable INVALID_ARGUMENT/2 while genuine file/integrity errors remain IO_ERROR/10. Time-zone metadata comes from a reported JVM property and client rules; storage dates remain unverified. Windows retries cover bounded access denial; the original intermittent deployment cause is unconfirmed. Resume improvement halves ordinary prefix scans but retains cumulative scaling limits.

Use a merge commit for this promotion. Version metadata is unchanged; release-please will propose the release version/changelog after promotion, with publication decided by the later release PR.

Closes #62
Closes #65
Closes #74
Closes #75

Refs #57
Refs #58

oxspan and others added 17 commits November 9, 2026 17:08
Document issue-based branch names and issue references in both contributor guides and pull request templates.
Report the verification limit in XLog search and capture output. Refs #57.
Clarify the profile warning for error text that can contain sensitive values. Refs #63.
Expose stored object hashes through object list --date. Refs #59.
Expose stored service and error dictionary text through the existing XLog search opt-in. Refs #60.
Add recorded timing, parent steps and network outcomes to live profiles. Refs #61.
Enforce Linux line and branch coverage thresholds and publish test reports. Refs #72.
Report actionable path errors for transaction capture and offline inspection, including regular-file ancestors, while preserving atomic creation, existing data and integrity failures.

Refs #75
Retry transient Windows access-denied atomic replacements and distinguish local file errors from Collector transport errors. Preserve committed checkpoints and document recovery limits.

Refs #58
Provide command-specific help, clean redirected and NO_COLOR output, and safe option-name diagnostics while preserving CLI execution contracts.

Refs #65
Collect committed transaction IDs during the existing validated traversal, preserving integrity and recovery while removing the extra prefix scan. Add reproducible local I/O measurements and document remaining scaling limits.

Refs #74
## Why

Profiles with repeated SQL can produce hundreds of kilobytes of JSON.
Add the single aggregate option retained by the issue's triage.

Refs #62

## Changes

`profile get --group-sql` and offline `inspect --group-sql` return SQL
hash groups with execution count, total/max elapsed milliseconds and
error count. Statements still require `--include-text`; unresolved or
conflicting dictionary text is explicit. The summary includes
decoded-step counts, lossless transaction IDs and original completeness
metadata. English and Korean guides explain the output and limits.

In a synthetic wire fixture with 3,000 repeated SQL steps, live output
with text fell from 667,803 bytes to 501 bytes. Default individual-step
output remains available.

## Validation

- Full Linux `coverageCheck verifyReleaseInputs`: passed; 266 executable
test cases, line coverage 95.38%, branch coverage 80.95% (90%/80%
minimums).
- Native Windows JDK 21: 10 new packaged-CLI scenarios and 6 aggregate
tests passed.
- Actual CLI comparisons against the previous build: byte-identical
default live/inspect output and capture files; previous strict reader
accepts new captures. Verified text redaction, unchanged capture bytes
after inspection, late-page corruption rejection, truncation,
unavailable profiles and command-option validation.
- Baseline and deliberate aggregation mutations demonstrated that the
new tests detect missing functionality, overflow and incorrect redacted
statement output.

Validation used synthetic local TCP fixtures; no external
Collector/Agent deployment was tested.

## Compatibility and risks

The opt-in payload declares `scouterctl.profile-sql-summary/v1`;
existing result envelopes, default outputs, capture formats and
exit/completeness semantics are preserved. Grouped output omits
individual chronology and non-SQL details. Size grows with distinct SQL
hashes; this does not reduce upstream transfer/decoding or prove trace
completeness. Duration totals count decoded steps and are not
transaction wall time.
## Why

Update the Java setup and GitHub Script actions while preserving the
artifact API required by local act runs.

## Changes

- Update `actions/setup-java` from v6.1.0 to v6.2.0 and
`actions/github-script` from v8 to v9.0.0, pinned to full commit SHAs.
- Keep the three act-only artifact uploads on v4.6.2. Hosted uploads
already use v7 and remain unchanged.
- Synchronize the branch with current develop, including SQL grouping.

## Validation

- act v0.2.97 workflow validation passed; both workflows and reusable
job graph resolve.
- Ran the actual comment script with v9's pinned execution function on
Node24 across seven scenarios, including creation/update, stale/closed
PRs, missing/oversized reports and report content treated as text.
External GitHub API calls were replaced locally.
- Confirmed upstream tags/SHAs, Node24 runtime and retained input
compatibility.

Full local act execution was unavailable because Docker Desktop's Linux
engine/WSL integration is unavailable. No environment settings were
changed. Latest-commit hosted CI must pass before merge. The develop
workflow skips Windows packaging and scale/release jobs; they are not
claimed as validated by this PR run.

## Compatibility and risks

Existing workflow permissions, conditions, script content and artifact
inputs are unchanged. Local act upload compatibility keeps its existing
v4 pin; the v9 script uses the supplied GitHub client and Node
built-ins, avoiding the removed CommonJS package import and
injected-name conflicts.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: oxspan <[email protected]>
When a Collector runs in KST, declaring the same instants as a Los
Angeles storage day can return an empty successful stream. `doctor` and
`collector status` now expose the reported JVM time-zone ID and the
CLI-derived UTC offset at the Collector's clock time, with explicit
provenance. Online XLog search, capture, counter history and counter
export reject differing offsets with `INVALID_ARGUMENT` before streams
or capture writes; equivalent offsets remain accepted, including
historical/DST interval checks.

The ID comes from the existing authenticated `SERVER_ENV` response, and
the offset uses client JDK rules. Missing/unparseable metadata remains
unknown. This does not certify the Collector's runtime default, TZDB or
fixed-day storage calendar. Storage-date verification remains
unverified, and capture schemas remain unchanged. English and Korean
guides explain these limits.

Validation: the complete Gradle regression suite passes, line coverage
95.35% and branch coverage 81.11%; release-input checks pass. The 22 new
packaged CLI/zone-rule cases also pass on native Windows Java 21.
Regression tests fail against the previous CLI and with deliberately
broken fallback/DST handling. Actual Java 8 Collector 2.21.3 KST/LA/KST
queries change from 3/0/3 records to 3/error2/3; v3 profile capture
bytes match the previous CLI and remain readable by its strict reader.

Refs #57
Windows reports a child of a regular file as `NoSuchFileException`.
Capture and inspect therefore advised creating a missing directory even
though a file blocked the path. Validate the lexical parent before
reporting a missing path, so nested file ancestors receive the existing
not-a-directory guidance while genuinely missing directories retain
their existing message.

The correction adds one line and preserves error codes, integrity
checks, existing contents and genuine filesystem failures. No tests were
changed, weakened or skipped. The existing three CapturePathTest cases
fail before the change and pass afterward on native Windows, covering
both capture formats and resume paths.

Validation:
- Complete hosted Linux and Windows suites and native command package
verification pass for the exact head in [run
43251124659](https://github.com/oxspan/scouterctl/actions/runs/43251124659).
Required PR CI also passes.
- Line coverage 95.35% and branch coverage 81.08%, with unchanged
thresholds.
- Both ZIP checksums verify; Linux and Windows share byte-identical JVM
payloads. The downloaded Windows executable passes version, redirected
help, argument-error, invalid/missing-path and read-only grouped
inspection checks; capture bytes remain unchanged.

Release-readiness [run
43247008118](https://github.com/oxspan/scouterctl/actions/runs/43247008118)
exposed the original Windows failure. No master promotion, release
publication or version change is included.

Refs #75
@github-actions

Copy link
Copy Markdown
Contributor

Linux CI: success · Commit df26cd3

Test results (Linux)

Total Passed Failed Errors Skipped
289 289 0 0 0

Test coverage

Metric Covered / total Coverage Minimum Result
LINE 2278 / 2389 95.35% 90.00% PASS
BRANCH 1371 / 1691 81.08% 80.00% PASS

Includes the test JVM and CLI subprocesses. Excludes optional real-Collector smoke tests.

Workflow logs and report artifacts

Updated on each CI run. Windows checks, when required, are reported separately in the workflow.

@oxspan
oxspan merged commit 30d6106 into master Nov 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant