Skip to content

Migrate sync-versions.sh → wfctl registry sync subcommand (consolidates Layer 3b + deferred I3 capability gate) #762

Closed
Closed
Migrate sync-versions.sh → wfctl registry sync subcommand (consolidates Layer 3b + deferred I3 capability gate)#762

Description

@intel352
intel352
opened on May 23, 2026

Context

workflow-registry/scripts/sync-versions.sh is bash + jq + gh CLI. workflow#758 added a strict-semver tag gate to it (PR #110, 940ecc405c) — but that regex is now duplicated with wfctl plugin validate-contract --for-publish (Layer 1). Two implementations of the same rule = drift risk.

Cycle-4 plan adversarial review flagged this as "tooling decision creep" (I1 in workflow-registry context); it was accepted pragmatically at the time to keep Layer 2 scoped down. Coming back to address it now.

Problems with the current bash sync

  1. Regex duplication between sync-versions.sh and wfctl plugin validate-contract --for-publish. Both define ^v\d+\.\d+\.\d+$. Future changes (e.g. allowing -rc.N once ParseSemver supports it) require touching two places.
  2. No test harness in workflow-registry repo. Layer 2 PR shipped without an automated test for the new gate — only an inline regex verification at PR time.
  3. Can't reach the next gate. The deferred binary-vs-file capability freshness gate (workflow#758 cycle 4-A1 I3) requires spawning the just-released plugin binary and calling its GetContractRegistry RPC, then diffing against the committed plugin.json.capabilities. Bash + jq cannot do that; wfctl already has plugin-spawn infrastructure (plugin/external/manager.go).
  4. Two source-of-truth problem. wfctl is the plugin-contract authority (it owns PluginManifest.Validate, ParseSemver, the gRPC client). Registry sync logic that knows about plugin manifests, version semantics, and capability shapes belongs in the same domain.
  5. Long-tail of bespoke shell logic — fetch_plugin_json shells gh api, downloads_match_version is a custom jq filter, release_downloads parses JSON via bash. Each has been a small regression hazard.

Proposed direction

New wfctl registry sync subcommand. Walks plugins/<name>/manifest.json in a checked-out workflow-registry repo, performs the same tag-fetch / version-compare / capability-fetch / manifest-rewrite logic as sync-versions.sh today, plus:

  • Strict-semver tag gate (single regex; same source as wfctl plugin validate-contract --for-publish).
  • (Future, deferred I3) --verify-capabilities flag: pull the just-released plugin tarball, exec the binary, call GetContractRegistry, diff against committed plugin.json.capabilities. Reject if stale.
  • (Future) --apply writes back; absent it's dry-run.

workflow-registry's GitHub Action becomes a one-line wfctl registry sync --apply (or whatever flag set).

Bundles with

This issue also bundles the related deferred work that all benefits from a Go-side migration:

Deferred from workflow#758

  • Layer 3b: apply plugin-version discipline to remaining 56 plugin repos (workflow#760). Mechanical fan-out; the canonical pattern shipped 2026-05-23. Audit list enumerated in Layer 3b: apply plugin-version discipline to remaining 56 plugin repos (workflow#758 follow-up) #760.
  • Binary-vs-file capability freshness gate at contract-check time (cycle 4-A1 I3). Natural fit for wfctl registry sync --verify-capabilities AND wfctl plugin validate-contract --for-publish --verify-binary (operator-side).
  • Full SemVer 2.0.0 prerelease support: requires concerted ParseSemver + sync-versions + wfctl install update. Naturally part of this scope if registry sync moves to Go.
  • Gap-repos: ~8 plugin repos lack release pipelines (agent, cms, compute, cloud-ui, data-protection, edge-compute, sandbox, waf). Separate from this migration but should be audited alongside.

Audit needed

Before brainstorming the design, an audit pass to inventory:

  1. Every shell-script in workflow-registry/scripts/ (not just sync-versions.sh) — what else lives there, what calls it?
  2. Existing wfctl plugin subcommand surface — what's the precedent for wfctl registry <verb> vs wfctl plugin registry-<verb>?
  3. workflow-registry's CI invocations of these scripts — what's the actual contract Github Actions consume?
  4. Any in-tree consumers besides CI (cron, manual runs by operators, downstream tooling)?
  5. The 56 Layer 3b repos: per-repo variance preview so the fan-out doesn't hit surprises mid-sweep (DO/AWS/GCP/Azure/github pilot already revealed: Version var name varies, main.go path varies, IaC vs non-IaC split, capabilities-shape variance).

Acceptance criteria

  • wfctl registry sync subcommand lands with table-driven tests
  • workflow-registry/scripts/sync-versions.sh deleted (or reduced to a single-line wfctl invocation)
  • Same strict-semver tag gate sourced from one regex constant in Go
  • --verify-capabilities (or equivalent) closes cycle 4-A1 I3
  • Layer 3b sweep (56 repos) completes
  • Gap-repos audited; per-repo path forward documented
  • Full SemVer 2.0.0 evaluated: ship the parser update OR explicitly defer with rationale

References

Activity

  1. intel352
    added 3 commits that reference this issue on May 24, 2026

    feat(wfctl): plugin registry-sync subcommand + shared release-grade s…

    7fcb535

    docs(plan): wfctl plugin verify-capabilities design (workflow#765)

    6743bc5

    feat(wfctl): plugin verify-capabilities subcommand (workflow#765) (#766)

    827158b

  2. intel352 commented on Jun 1, 2026

    @intel352
    intel352
    on Jun 1, 2026
    ContributorAuthor

    Audit result: still relevant, partially shipped.

    Verified against latest origin/main: PR #763 shipped wfctl plugin registry-sync, the shared publish-grade semver regex, and docs. Later wfctl plugin verify-capabilities work also exists. Remaining scope is still real: registry-sync --verify-capabilities still prints not implemented, registry-sync core/readme native ports are still pending, and the Layer 3b/c sweep remains tracked by #760.

  3. intel352 commented on Jun 2, 2026

    @intel352
    intel352
    on Jun 2, 2026
    ContributorAuthor

    Updated issue #762 cascade status after the registry-sync follow-up.

    Completed/verified:

    • Merged workflow PR fix: keep registry sync from mutating builtins #835 (fix: keep registry sync from mutating builtins) at 7a9dc544f980c821dc3af65b4adf30c05fbb0e45.
      • Default wfctl plugin registry-sync now skips builtin/core manifests.
      • wfctl plugin registry-sync core now treats stray downloads on core manifests as drift and removes them with --fix.
      • Added regressions for builtin skip and downloads-only drift.
    • Published workflow release v0.69.7; release workflow 26794682772 completed successfully and assets were verified.
    • Closed workflow-registry PR feat: add step.auth_validate pipeline step #190 because the branch preserved stale builtin downloads from the pre-v0.69.7 generator bug.
    • Opened clean replacement workflow-registry PR feat: add step.raw_response pipeline step for non-JSON HTTP responses #191 from current origin/main, fixed Copilot checksum feedback, all checks green, merged at d2e266701bc2d015ac2b7af2055b6ba0f2fd253d.
    • Verified no remaining open PRs in workflow-registry or workflow-plugin-digitalocean.

    Current open workflow PR state:

    Remaining #762 work to continue next:

    • Continue issue-driven iteration beyond the completed registry-sync/core/readme cascade, especially remaining plugin fanout/gap repos and any still-open verified issue scope not covered by the merged PRs above.

  4. intel352 commented on Jun 2, 2026

    @intel352
    intel352
    on Jun 2, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Dependency PR cleanup from the previous status note is now complete.

    Merged after fixing module tidy drift and verifying CI/review state:

    Verification notes:

    • Fixed example module tidy drift.
    • Fixed cmd/wfctl/testdata/verify_capabilities/* fixture module tidy drift exposed by readonly fixture builds.
    • Ran focused verify-capabilities tests locally before each push.
    • CI was green before admin merge; no review threads remained.
    • Copilot review could not be requested because GitHub reports copilot-pull-request-reviewer is not a collaborator on this repository.

    Leaving #762 open because the broader registry-sync/fanout/gap-repo scope remains active.

  5. intel352 commented on Jun 2, 2026

    @intel352
    intel352
    on Jun 2, 2026
    ContributorAuthor

    Closing this as completed after re-audit on 2026-06-02.

    Verified current state:

    • wfctl plugin registry-sync exists on workflow origin/main (c5d6861) with native default, core, and readme modes.
    • --verify-capabilities is implemented, not stubbed: it selects the current-platform release asset, downloads via gh release download, extracts tarballs, locates the binary, and reuses the wfctl plugin verify-capabilities manifest check with registry-name alias handling.
    • registry-sync core removes stray downloads from builtin/core manifests and has regression coverage.
    • workflow-registry origin/main (23d3db2) now has scripts/sync-versions.sh, scripts/sync-core-manifests.sh, and scripts/generate-readme.sh reduced to thin wfctl plugin registry-sync compatibility wrappers. CI uses wfctl plugin registry-sync, core, and readme directly.
    • Layer 3b is complete via workflow#760, which is closed with sweep notes.
    • Gap repo audit: active repos workflow-plugin-agent, workflow-plugin-cms, workflow-plugin-compute, and workflow-plugin-edge-compute all have .github/workflows/release.yml; workflow-plugin-cloud-ui, workflow-plugin-data-protection, workflow-plugin-sandbox, and workflow-plugin-waf are no longer resolvable as repos.
    • Full SemVer 2.0.0 prerelease support remains intentionally deferred in the design/docs; current shared release-grade regex is still the flat vMAJOR.MINOR.PATCH gate used by both validate-contract and registry-sync.

    Fresh verification:

    • `GOWORK=off go test ./cmd/wfctl -run
  6. intel352
    closed this as completedon Jun 2, 2026

  7. intel352 commented on Jun 2, 2026

    @intel352
    intel352
    on Jun 2, 2026
    ContributorAuthor

    Verification addendum for the close note above (the previous comment's test-command line was truncated by shell quoting):

    Fresh checks run on workflow origin/main (c5d6861):

    • GOWORK=off go test ./cmd/wfctl -run 'TestPluginRegistrySync|TestPluginVerifyCapabilities' -count=1 -> ok github.com/GoCodeAlone/workflow/cmd/wfctl 1.984s
    • GOWORK=off go test ./cmd/wfctl -run 'TestPluginRegistrySync' -count=1 -> ok github.com/GoCodeAlone/workflow/cmd/wfctl 1.067s

    Also verified workflow-registry origin/main (23d3db2) has no open issues/PRs, and workflow has no open PRs.

    I also attempted the broader GOWORK=off go test ./cmd/wfctl -count=1; it produced no output for over two minutes and was stopped rather than left running. I am not claiming a broad package pass from that command. The focused registry-sync / verify-capabilities coverage above is the evidence for closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions