feat: system attestation types#225
Closed
mintlify[bot] wants to merge 13 commits into
Closed
Conversation
Contributor
Author
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
The page was moved from administration/managing_custom_attestation_types/ overview.md to attestation_types/custom/managing.mdx as part of consolidating the custom-type docs. The redirect is already in place in config/redirects.json and the navigation entry was removed; the old file was reintroduced by the rebase and is now redundant. Delete it so visitors only resolve the new path. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Replace the single GitHub example with an AccordionGroup covering all four supported providers (GitHub / GitHub Enterprise, GitLab / self-hosted GitLab, Bitbucket Cloud, Azure DevOps). Each accordion has a copy-pasteable example and a flag table noting which flags are required, which auto-default in CI, and which only apply to self-hosted / enterprise variants. Also add a "What gets recorded" subsection clarifying that auth tokens are never stored in the attestation — the flags are CLI inputs, not attestation content. The pull_requests array shape is identical across providers. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
System-type compliance is now described in customer-observable terms, not implementation. Each per-type page states what the compliance check actually is for that type: - system:sarif — schema-driven (records facts, no built-in opinion) - system:junit — every suite has zero failures and zero errors - system:jira — at least one referenced issue, all exist - system:sonar — quality-gate status is OK - system:pull_request — at least one PR for the commit - system:generic — caller-supplied is_compliant flag The legacy-type compliance rules preserve the behaviour of the dedicated commands they replace (kosli attest junit etc.), so existing customers can migrate to kosli attest system <type> without changes. For new system types (starting with system:sarif) compliance is schema-only; opinions live in kosli evaluate + rego on a separate attestation. Removes the jq mentions and "structural only" framing from customer-facing copy. jq remains as an optional internal implementation detail for legacy types — never surfaced to customers. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Switch the customer-facing model to schema-only compliance for every system
type. The earlier "legacy types carry their existing compliance behaviour
forward" framing is gone — system types record facts; opinions live in
kosli evaluate + rego on a separate attestation.
Migration from a legacy kosli attest <type> command does NOT preserve the
built-in verdict (e.g. kosli attest junit was non-compliant on any failed
test; system:junit is compliant once results are recorded). This is loud
on every per-type page and in a new dedicated migration guide that ships a
copy-paste rego policy for each legacy command to reproduce its verdict
via kosli evaluate.
Changes:
- system/overview.mdx: compliance section is now "schema-validated only".
Adds a Warning callout pointing at the new migration page.
- system/{junit,jira,sonar,pull_request}.mdx: compliance sections rewritten
to schema-only with a pointer to the migration page anchor.
- system/sarif.mdx and system/generic.mdx tightened for consistency.
- attestation_types/overview.mdx and custom/overview.mdx comparison
tables: System row simplified to "schema-validated only; use kosli
evaluate for opinions".
- system/catalogue.mdx: legacy types are now described as deprecated,
not the recommended path, with a link to the migration guide.
- system/legacy_migration.mdx (NEW): per-type migration guide. For each
legacy command, diff-style attest call migration plus a copy-paste rego
policy reproducing the legacy compliance rule, ready to run via
kosli evaluate trail. Covers snyk, junit, sonar, jira, pullrequest_*,
generic.
- config/navigation.json: new page registered under System attestation types,
between overview and the available-types subgroup.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The legacy-command migration content lived under attestation_types/system/. Move it to tutorials/ in the "Attesting" group where it belongs — it's a workflow, not a type reference. While moving, restructure for the actual customer use case: - Reframe as a four-step workflow: replace the attest call, write the rego, run kosli evaluate trail with --output json, and record the verdict as a system:generic attestation with --compliant/--attachments/--user-data. Step 4 is what the old page was missing — without it the rego verdict doesn't end up on the trail at all. - Show the kosli evaluate output (RESULT/VIOLATIONS) so readers know what they should see in CI. - Per-type recipes go in collapsible accordions: snyk, junit, sonar, jira, pullrequest_*, generic. Each accordion holds just the diff and the type-specific rego; the workflow shape is shared at the top so we don't repeat steps 3 and 4 six times. Also: - Add a redirect from the old /attestation_types/system/legacy_migration path to the new /tutorials/migrating_to_system_types so any external link still resolves. - Update every per-type page's cross-reference (jira, junit, sonar, sarif, pull_request, generic), plus catalogue and the overview Warning. - Rewire navigation.json: drop the old page from the system types group; add the new tutorial between attest_sarif and attest_snyk under the Attesting group. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Jon's review feedback: every rego recipe hardcoded the attestation name
(e.g. "security-scan", "unit-tests") inside the policy, but the name
varies per customer. Same pattern as the OPA tutorial's
data.params.max_high / max_medium thresholds.
Read attestation_name := data.params.attestation_name at the top of each
policy and reference [attestation_name] in the lookups. One rego file
works regardless of what --name the customer set on the system attestation.
If the param is missing the lookup is undefined, the compliance condition
fails to fire, and allow stays false — correct fail-safe behaviour.
Step 3 of the general workflow now shows --params '{"attestation_name":
"security-scan"}' on the kosli evaluate trail call, and every accordion's
"run steps 3 and 4" line tells the reader which value to pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Jon's review: --artifact-type docker my_company/backend:latest is valid but hard to scan because the positional sits in the middle of the flag values. Convention elsewhere on the page (the kosli attest artifact example just above this one) puts the positional right after the command. Match it. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…ypes Jon's review: the system-types overview said system types "know the shape of the input, parse it, and render it in the trail" — true on the server side but missing the bigger client-side win. The Kosli CLI takes the raw output from your tool (JUnit XML directory, SARIF file, Sonar project key, Jira API, git provider API), parses it, posts the structured attestation, AND uploads the source file to the Evidence Vault as an attachment. One command, no glue scripting. Custom types require you to do that work yourself before attesting. Updates four pages so the contrast is visible everywhere a customer might land: - attestation_types/system/overview.mdx — adds a "First-class CLI support for the source format" bullet and a contrast paragraph about custom types parsing being the caller's problem. - attestation_types/overview.mdx — System Card description rewritten; comparison table grows a "Data preparation" column. - attestation_types/custom/overview.mdx — comparison table mirrored to carry the same "Data preparation" row. - getting_started/attestations.md — System types accordion leads with the CLI-parsing-and-vault story; Custom types accordion adds an explicit contrast paragraph noting the caller produces the JSON. Also picks up the "structural" → "schema-validated" wording fix while in the same block, and adds a link to the per-type migration tutorial we just shipped. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Already implied by the rest of the page — and the customer experience is better served by letting the table speak for itself. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Surface the default value on the per-type page so callers don't get caught out — kosli attest system generic with no --compliant flag produces a compliant attestation, matching the legacy kosli attest generic default. Also note the --compliant=false vs --compliant false gotcha that bites with Cobra boolean flags (same warning already exists in the migration tutorial and the OPA tutorial). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Jon's review: kosli evaluate trail has a --no-assert flag that makes the command exit 0 even when the policy denies, so the surrounding "|| true" and stderr redirect are unnecessary. The new shape is cleaner and surfaces real stderr if something goes wrong (vs. silently swallowing it). Also add a one-sentence explanation of why --no-assert is needed for the two-step "evaluate then attest" flow — the deny is captured in the JSON report instead of as an exit code, which step 4 reads from. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Contributor
|
Closing as not planned. We're parking the system-types initiative for now — no immediate customer or business driver makes it the right thing to ship over other work. Nothing here is wasted:
If a customer driver emerges that justifies prioritising this, reopen the master ticket #5647 and the sub-issues; the docs branch is mergeable as a starting point. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE YET OR APPROVE PLEASE - DOCS DRIVEN DEVELOPMENT
Mirror of
feat/system-attestation-typesopened through Mintlify so the preview build runs.No content changes — this branch contains the same diff as the original
feat/system-attestation-typesbranch, just opened via the Mintlify-authored flow so the Mintlify preview check is triggered.Original PR: #224