Skip to content

ci: 👷 Enforce coverage and report test results on PRs - #73

Merged
oxspan merged 3 commits into
developfrom
feature/72-ci-test-coverage
Nov 11, 2026
Merged

oxspan merged 3 commits into
developfrom
feature/72-ci-test-coverage

Conversation

@oxspan

@oxspan oxspan commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Why

Passing tests previously provided no coverage report, minimum coverage check, or PR-visible result summary. Refs #72.

Changes

  • Add ./gradlew -Pcoverage=true coverageCheck with JaCoCo, enforcing 90% line and 80% branch coverage. Merge test JVM data with separate CLI subprocess execution files without changing CLI output.
  • Reuse the Linux CI test run for coverage/package verification, publish counts and thresholds in Actions, and retain HTML/XML reports for 14 days, including threshold failures. Missing JVM data or empty report counters fail verification.
  • Create one bot comment per same-repository PR with test totals/passed/failed/errors/skipped, line/branch coverage and minimums, the CI result, commit, and workflow/artifact link. Re-runs update the bot comment; outdated commits and closed PRs are skipped. Missing reports are explicitly unavailable.
  • Isolate comment write permission in a reporting job that reads a Markdown artifact without checking out or executing PR code. Fork and Dependabot PRs retain Actions summaries/artifacts without comments.
  • Add subprocess instrumentation regression coverage and 16 dated-object response validation cases. Document usage and reporting behavior in the testing and English/Korean contribution guides.

Validation

  • Latest hosted CI passed on 2337502, including the comment job. Published bot comment: 228 tests passed, 95.55% lines (2060/2156), 80.21% branches (1228/1531).
  • Local coverageCheck verifyReleaseInputs passed; coverage-disabled command distribution tests passed (5 tests).
  • Verified failures for missing test JVM data, missing CLI data, empty XML counters and coverage below the branch threshold. Threshold failure preserves its report/FAIL summary.
  • Reporter fixture checks passed for multi-suite counts, failures/errors/skips, absent/malformed/empty reports, comment creation/update, leaving user comments intact, stale/closed PRs and missing/oversized artifacts.
  • actionlint and whitespace checks passed. Local act was attempted but could not fetch actions/checkout due to an authentication error; hosted CI completed successfully.
  • Native Windows and real Collector/Agent/Spring smoke checks were not run for this develop PR.

Compatibility and risks

Instrumentation is test-only and opt-in locally. Normal Linux CI enables it; ordinary tests and Windows checks keep coverage disabled. Historical release-asset repair skips the new coverage gate. No production CLI options, output formats, runtime dependencies or capture formats change. Codecov comparisons and required merge checks remain follow-up work.

@github-actions

Copy link
Copy Markdown
Contributor

Linux CI: success · Commit 2337502

Test results (Linux)

Total Passed Failed Errors Skipped
228 228 0 0 0

Test coverage

Metric Covered / total Coverage Minimum Result
LINE 2060 / 2156 95.55% 90.00% PASS
BRANCH 1228 / 1531 80.21% 80.00% PASS

Includes the test JVM and CLI subprocesses. Excludes optional real-Collector smoke tests.

Workflow logs and report artifacts

Updated on each CI run. Windows checks, when required, are reported separately in the workflow.

@oxspan oxspan changed the title ci: 👷 Enforce line and branch coverage in Linux CI ci: 👷 Enforce coverage and report test results on PRs Nov 10, 2026
@oxspan
oxspan marked this pull request as ready for review November 11, 2026 07:14
@oxspan
oxspan merged commit 091d805 into develop Nov 11, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant