Repository navigation
Database-backed plugin directory - #583
Conversation
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
rebuildRepo now scopes its final write to the build's id and refuses it (rather than falling back to Create) if Delist removed the row during the in-flight network build, and checks nameTaken before writing a renamed plugin's document so a collision during rebuild is ErrNameTaken with the previous document kept, not a raw DB error. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
RejectDirectoryRepo ignored BindJSON's error, which had already written a 400 on decode failure (including on an empty body, which was meant to mean "no reason"); the handler then wrote 200 on top of it. Switch to ShouldBindJSON, only treat a malformed non-empty body as an error, and switch AddDirectoryRepo to ShouldBindJSON for consistency. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…er GC Drop the gorm text-type tag on Build.Doc/LastError (MySQL's 64 KiB text cap can't hold a big README+changelog+release-notes doc). Add wasm.Options.NoCache so validating a public submission doesn't pin its compiled module in the process-wide wazero cache forever. Bound the directory's IP limiter map with a periodic sweep of expired addresses (unbounded over IPv6). Set the 429 Content-Type before flushing the header so it isn't sniffed. Use Limit(1).Find instead of First for Detail() so crawler hits on unknown plugin names stop spamming gorm ERROR logs. Cap GitHub release pages at 10 and rendered README/CHANGELOG HTML at 1 MiB so one pathological repo can't cost unbounded API calls or storage. Fix a stale renderDetail comment. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
README: note that README/changelog/release-note HTML is rendered by GitHub's markdown API and shown as-is, including in the admin's pending card. PLUGIN_CONTRACT: "the tool" is now "submission" (there is no separate tool). Spec: submitter_ip is kept for the admin (rate limiting is in-memory, not DB-backed); the wire types live in plugins/directory/registry and are aliased in plugins/directory, only Fetcher moves; requireDirectory's 503 is about the plugin not being wired/initialised, not about enabled=false (admin curation works while disabled, for seeding); expand the seeding checklist (site_url, github_token, refresh_minutes, a manual Directory-tab pass); rename Out of scope to Out of scope / accepted risks and record the unbounded pending-submission count, the 90s lock-hold risk from dropping the Docker fence, and the CSRF follow-up's new submit-then-approve angle. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
There are concrete security/UX issues to address in the new directory admin API and submit parsing (notably Content-Type enforcement for CSRF hardening and case-insensitive GitHub URL parsing).
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 3
Open (3)
What changed in this PR
This PR converts the built-in directory plugin from a remote-index mirror into a database-backed plugin registry that validates GitHub repos in-process, stores curated entries/build artifacts in new DB tables, and serves /plugins pages + JSON from that data. It also adds admin APIs/UI for approvals and moves the installer’s directory fetcher into plugin/installer.
Changes:
- Implement DB-backed directory registry with submission flow (
/plugins/submit), validation/build pipeline, and scheduled refresh. - Add admin directory API endpoints and a new Admin → Plugins → Directory tab for curation.
- Move installer
Fetcherintoplugin/installer, add wasmNoCacheoption for validation, and update docs/README.
| File | Description |
|---|---|
| themes/default/templates/admin_settings.html | Add password input rendering for plugin settings. |
| themes/default/templates/admin_plugins.html | Add “Directory” admin tab and JS to manage pending/approved/rejected repos. |
| README.md | Rewrite plugin directory documentation to match DB-backed flow. |
| plugins/directory/templates/submit.html | New public submission form template with honeypot. |
| plugins/directory/templates/listing.html | Update listing copy and empty-directory messaging; link to submit page. |
| plugins/directory/submit.go | Implement submit parsing, honeypot, rate-limit/busy handling, and rendering. |
| plugins/directory/submit_test.go | Tests for repo parsing and submit handler behavior/status codes. |
| plugins/directory/store.go | Add directory_repos / directory_builds models + index encoding helpers. |
| plugins/directory/store_test.go | Persistence + encoding tests for repo/build storage and index generation. |
| plugins/directory/service.go | Core registry service: submit/add/approve/reject/delist/rebuild/refresh/index cache. |
| plugins/directory/service_test.go | Service behavior tests including races, rebuild semantics, refresh, and listing. |
| plugins/directory/render.go | Add submit page renderer; adjust detail renderer signature/docs. |
| plugins/directory/registry/validator.go | Add WasmValidator + FakeValidator and sha helper. |
| plugins/directory/registry/validator_test.go | Validator tests (identity, garbage, cancelled context, fake validator). |
| plugins/directory/registry/validate.go | Port/implement end-to-end repo validation (ValidateEntry, LatestVersion). |
| plugins/directory/registry/validate_test.go | Validation tests covering tag rules, missing files/assets, mismatches, etc. |
| plugins/directory/registry/source.go | Implement GitHub REST client (Source) with paging, limits, markdown rendering. |
| plugins/directory/registry/source_test.go | HTTP fake tests for GitHubSource correctness and bounds. |
| plugins/directory/registry/manifest.go | Manifest schema + validation rules (name/license/runtime/hosts/min version). |
| plugins/directory/registry/manifest_test.go | Manifest validation tests (defaults, host rules, errors). |
| plugins/directory/registry/build.go | Build detail docs (README/changelog/releases), size caps, stars best-effort. |
| plugins/directory/registry/build_test.go | Build behavior tests, URL shaping, rendered-size limits, error propagation. |
| plugins/directory/limiter.go | In-memory IP limiter with sweep to bound memory growth. |
| plugins/directory/limiter_test.go | Limiter behavior and sweep pruning tests. |
| plugins/directory/index.go | Replace local wire types with aliases to registry types. |
| plugins/directory/directory.go | Make directory plugin the registry: migrations, service wiring, routes, refresh job. |
| plugins/directory/directory_test.go | Integration tests for directory plugin pages, JSON, submit, and hosted behavior. |
| plugin/wasm/wasm.go | Add NoCache option to skip shared compilation cache for validation loads. |
| plugin/wasm/wasm_test.go | Test NoCache load path. |
| plugin/installer/installer.go | Switch installer to use local Fetcher and add directory_hosted to status. |
| plugin/installer/installer_test.go | Update tests for moved fetcher. |
| plugin/installer/fetcher.go | Move fetcher implementation into installer package; keep types from directory pkg. |
| plugin/installer/fetcher_test.go | Update package declaration for moved fetcher tests. |
| goblog.go | Wire installer fetcher move; wire directory plugin into admin; add directory API routes. |
| docs/superpowers/specs/2026-09-20-db-backed-plugin-directory-design.md | Add design/spec doc for DB-backed registry. |
| docs/PLUGIN_CONTRACT.md | Add/port plugin publishing contract documentation. |
| admin/plugins.go | Include directory_hosted in /api/v1/plugins/status. |
| admin/plugins_test.go | Update harness for installer fetcher move and admin pointer usage. |
| admin/directory.go | New admin directory API handlers (/api/v1/directory/...). |
| admin/directory_test.go | Tests for directory API lifecycle, error mapping, and plugin status integration. |
| admin/admin.go | Add Directory *directory.Plugin field to Admin wiring. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…-directory # Conflicts: # plugins/directory/directory_test.go
…CSRF test for approve ParseRepo now accepts any capitalisation of github.com and ErrBadRepo mentions the bare owner/repo form. Password-typed plugin settings render empty in every theme (the stored secret no longer reaches the page) and an empty password is skipped on save, so blank means keep. The Content-Type check from #571 already covers the directory endpoints; csrf_test.go now pins the cross-site /approve case. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

Summary
directoryplugin is now the registry: repos are submitted at/plugins/submit(plain HTML form, honeypot, 5/h/IP, one validation at a time, 90 s budget), validated in-process (wasm.LoadBytesin the Extism sandbox, no compilation-cache retention) and queued; admins approve / reject / add / rebuild / delist under Admin → Plugins → Directory;/plugins,/plugins/<name>,/plugins/<name>.jsonand/plugins/index.jsonare served from two new tables (directory_repos,directory_builds).plugins/directory/registryports the registry repo's validation/build (GitHub overnet/http, no new deps): manifest rules,ValidateEntry,BuildRepo,LatestVersion. Scheduled refresh only re-downloads when a repo's latest tag changed; a failed rebuild keeps the previous document and recordslast_error./api/v1/directory/…;/api/v1/plugins/statusgainsdirectory_hosted.Fetchermoves toplugin/installer(no behaviour change for other goblogs).index_urlremoved;refresh_minutesdefault 360 (min 15); optionalgithub_token(password input).docs/superpowers/specs/2026-09-20-db-backed-plugin-directory-design.md(+ plan),docs/PLUGIN_CONTRACT.mdmoved in from the registry repo, README section rewritten.Follow-ups (not in this PR)
goblogplatform/pluginsafter this is released and goblog.live is seeded (plan Task 14).submit/indexplugin names, per-release notes size cap; Harden admin API against CSRF: SameSite on the session cookie and Content-Type check on JSON binders #571 now also covers "submit publicly, then CSRF an admin into /approve".Test plan
go test ./...green (new tests: registry ×4 files, store/limiter/service/directory/submit, admin directory API, wasm NoCache)goblog-plugin-helloandgoblog-plugin-scholarvia/plugins/submit(both queued), approved hello →/plugins,/plugins/hello,/plugins/hello.json,/plugins/index.jsoncorrectsite_url, setgithub_token,refresh_minutes=360, seed hello + scholar🤖 Generated with Claude Code