P2 [CRF-1] Missing descriptions fail generation, but a missing DisplayName case renders a title-cased guess and ships silently, so the generator does not prevent the exact bug this PR fixed by hand. (Knov P2, Meruem P3, Mafuuu Note)

render enforces one half of the metadata contract. A known experiment with no description returns an error at line 166 and generation fails loud. A known experiment with no case in DisplayName() falls through to the default at codersdk/deployment.go:5248, which title-cases the key (mcp-tool-search -> "Mcp Tool Search", and underscore keys are worse: nats_pubsub -> "Nats_pubsub"). render writes that verbatim and succeeds.

This is not hypothetical: correction #2 in the PR body is precisely this failure for ExperimentMCPToolSearch, which proves the class is live and recurring. TestRenderCoversEveryKnownExperiment does not catch it (see CRF-2). The description path is fail-closed; the display-name path is fail-open for the same class of omission.

Make the two paths symmetric: have render reject an experiment whose DisplayName() equals the default title-casing of its key, or store display names beside descriptions so both are enforced by the same error. The higher severity reflects that the automation added here leaves open the exact hole it was built to close.

🤖

Accepted. Generation now parses explicit DisplayName switch cases and fails when a known experiment lacks one. Resolved in 553b35a.

Verified fixed in 553b35a. render now rejects any known experiment absent from readDisplayNames, and the display-name path fails closed like the description path. Two residual gaps on this same guard are raised as new findings (CRF-19: presence checked, not authored non-empty content; CRF-17: the negative path is untested).

🤖

P3 [CRF-2] The display-name assertion is tautological and cannot fail on the quality it appears to guard. (Meruem P3, Bisky Note, Netero Note)

TestRenderCoversEveryKnownExperiment asserts strings.Contains(page, exp.DisplayName()) for every known experiment. Since render builds each row with exp.DisplayName(), the page contains that exact string by construction whenever the row renders, and a row that fails to render already errors via the missing-description path. The check only restates that the description guard fired; it gives zero coverage of display-name correctness. This is why the fail-open default in CRF-1 passes CI.

Fix: assert that no rendered display name equals the title-cased-key fallback, or replace this line with a golden-file comparison of the full table so both key and display name are pinned.

🤖

Accepted. Replaced the tautological assertion with direct explicit-case coverage and a missing-display-name failure test. Resolved in 553b35a.

Verified fixed in 553b35a. The tautological assertion is gone; TestRenderCoversEveryKnownExperiment now asserts the key literal and display-name authorship moved to TestReadDisplayNames against the real source.

🤖

P3 [CRF-3] The generator publishes raw constant comments as the Description column, so an internal maintainer note ships as user documentation. (Ryosuke P3, Leorio P3, Chopper P3, Pariston P3, Mafuuu P3, Mafu-san P3, Netero P3, Knov P3)

The page's stated job is to tell an operator what each experiment does. For auto-fill-parameters the cell reads "This should not be taken out of experiments until we have redesigned the feature." That is a team decision about lifecycle; an operator deciding whether to enable the key learns nothing about the feature, which is the gap this page was built to close. The example row ("This isn't used for anything.") is a milder, arguably honest instance for a placeholder. The generator's guard fails on an absent comment but cannot tell a description from an internal note, so presence is enforced and fitness is not.

The PR body acknowledges this and says a follow-up was "filed," but no ticket is linked in the diff. Under the no-follow-up assumption the misleading rows ship as the official public description. This needs a human decision: link the tracking ticket, rewrite the two constant comments in codersdk/deployment.go in this PR (keeping any lifecycle note as a separate non-trailing comment so it stays out of the rendered cell), or explicitly accept the gap. See CRF-8 for the structural fix.

🤖

The missing-ticket premise is false: DOCS-905 exists, is linked to DOCS-139, and specifically tracks these two comments. I kept the wording follow-up there because behavior verification needs product ownership. 553b35a makes missing descriptions fail closed.

Accepted as a ticketed deferral (DOCS-905). The two internal-note descriptions (auto-fill-parameters, example) still ship on the rendered page; tracking the wording change under product ownership is a reasonable disposition.

🤖

P3 [CRF-4] The hand-written wildcard note restates a live fact the generated page derives, so it drifts the moment ExperimentsSafe is populated. (Hisoka P3, Ryosuke P3, Knov P3)

The generated experiments.md computes the wildcard's effect from codersdk.ExperimentsSafe at run time: non-empty prints the safe list, empty prints "the wildcard enables nothing." This page hardcodes the empty case in prose: "no experiment currently carries that mark, so name the experiments you want instead." Nothing regenerates this file. The whole PR exists because hand-maintained experiment docs drift from source; this change fixes the drift in the reference page and reintroduces it here, on a security-relevant opt-in, with no mechanical guard. The day someone marks an experiment safe, make gen updates the reference and this guide asserts the opposite.

Keep the durable clause (the wildcard enables only the experiments Coder marks safe, not all) and drop the time-bound one, pointing at the generated reference, which already sits two lines down.

🤖

Accepted. Removed the time-bound empty-safe-set claim; the generated reference remains the source for the current wildcard set. Resolved in 553b35a.

Verified fixed in 553b35a. feature-stages.md dropped the time-bound empty-safe-set clause and links to the generated reference, which owns the dynamic wildcard fact.

🤖

P3 [CRF-5] Descriptions and display names are interpolated into the Markdown table without escaping |, so a comment containing a pipe corrupts the row. (Hisoka P3, Mafuuu P3, Meruem/Ryosuke/Pariston Note)

fmt.Fprintf(&b, "| %s | \%s` | %s |\n", exp.DisplayName(), key, desc)trusts that no cell contains the delimiter.sentencecollapses newlines viastrings.Fields, so multi-line comments are safe, but a literal |survives. No current comment carries one, so this is latent. The repo already knows the fix and applies it in sibling generators:strings.ReplaceAll(value, "|", "\|")inscripts/gotestsummary/main.go, and scripts/clidocgen/gen.gojoins with|. This generator diverges from that pattern. markdown-table-formatterruns afterward, so the worst case may be a gen-time break rather than a shipped defect, but the established pattern is to escape. Escape|indesc` and defensively in the display name before writing the row.

🤖

Accepted. Table-cell values now escape Markdown pipe delimiters, with coverage. Resolved in 553b35a.

Verified fixed in 553b35a. markdownCell escapes | on both display name and description, covered by TestRenderEscapesMarkdownTableCells.

🤖

Nit [CRF-11] The readDescriptions doc repeats the AST-parsing rationale already stated in the package doc. (Gon)

The package doc already says the descriptions live only as trailing comments and are read from the syntax tree; the function doc restates the same why. Put the rationale once at the package doc and let the function doc state only its contract: readDescriptions returns the trailing comment on each Experiment constant, keyed by the experiment's string value.

🤖

Accepted. Reduced readDescriptions to its contract; the package comment retains the AST rationale. Resolved in 553b35a.

Verified fixed in 553b35a. readDescriptions is trimmed to its contract and the AST rationale stays on the package doc.

🤖

Nit [CRF-12] commentText returns a formatted sentence, not raw comment text, so the name misleads. (Gon)

The function runs its result through sentence(), so callers get capitalized, single-line, period-terminated text. The name promises the raw comment. commentSentence, or moving the sentence() call to the caller, makes the return value match the name.

🤖

Accepted. Renamed commentText to commentSentence. Resolved in 553b35a.

Verified fixed in 553b35a. Renamed to commentSentence. Note the sibling readDisplayNames introduced the same name-vs-return mismatch this rename fixed; raised as CRF-21.

🤖

Note [CRF-13] The intro hardcodes example and workspace-usage as sample keys inside a generator whose purpose is to prevent doc drift. (Gon, Ryosuke)

Both are real experiment keys today, so the example is correct now. If either is renamed or removed, the generated intro shows an invalid key while every other line on the page stays source-derived and correct. Not worth a guard now; noting so a future key removal remembers this line.

🤖

Accepted. Replaced the hard-coded sample keys with . Resolved in 553b35a.

Verified fixed in 553b35a. The intro uses a generic placeholder.

🤖

Nit [CRF-14] The wrapped case in TestReadDescriptions claims to test comment wrapping but its comment is a single trailing line. (Bisky)

ExperimentWrapped ... // Spans a comment is one trailing line; a trailing comment cannot span lines. The multi-line collapse it implies is already tested in TestSentence ("collapses a wrapped comment"). Rename it to reflect what it checks (a plain trailing comment) or drop it as redundant with the example row.

🤖

Accepted. Renamed the case to trailing and made it describe a plain trailing comment. Resolved in 553b35a.

Verified fixed in 553b35a. The case is renamed to trailing and describes a plain trailing comment.

🤖

Note [CRF-15] The table is sorted by experiment key, so the leading "Experiment" (display name) column is not alphabetical. (Chopper)

Rows are ordered by slices.Sort(known) on the key string, so a reader scanning the first column (e.g. "SMTP and Webhook Notifications" under the notifications key) will not find the display names in order. Sorting by key is deterministic and defensible; at 13 rows the cost is small. Worth knowing, not worth changing.

🤖

Declined. Key order is the stable operator-facing identifier order and matches the page's command input; at 13 rows, a separate display-name sort does not improve lookup enough to outweigh that consistency.

Defense accepted; closing. The panel judged 10/10 that sorting by key keeps the Key column (what an operator types into --experiments) alphabetical, and the unsorted display-name column is bounded friction at 13 rows. No re-raise.

🤖

P3 [CRF-19] The display-name gate checks that an explicit case exists, not that it returns an authored non-empty name, so it is narrower than the description gate it mirrors. (Knov P3, Hisoka Note)

Two residual gaps on the CRF-1 fix:

1. Empty return. The description gate fails on desc == "" (line 220), but the display-name gate only checks !displayNames[key] (line 224). The rendered value comes from exp.DisplayName(), which nothing checks. A future case ExperimentX: return "" satisfies displayNames["x"] == true, so render proceeds and ships a blank leading cell silently, the same class of silent bad metadata CRF-1 set out to prevent. The fix mirrors the description gate: if !displayNames[key] || displayName == "".

2. Over-broad match. readDisplayNames walks the entire DisplayName FuncDecl body and marks a constant satisfied if it appears in any CaseClause.List. Today DisplayName is one flat switch, so intent and check coincide. If it ever grows a second switch, a guard clause, or a nested switch that mentions an experiment constant for any reason, that constant is marked "has a display name" and render stops failing for it, re-opening the CRF-1 regression. Scoping collection to the SwitchStmt whose tag is the receiver ident would make the check mean what it reads. Both are latent; the empty-return case would at least surface in the docs diff, the scope case would not.

🤖

P3 [CRF-17] No test exercises readDisplayNames on a source where a constant lacks its DisplayName case, so the CRF-1 fail-closed guarantee rests on an untested negative path. (Bisky)

The two halves of the guarantee are each tested but never meet. TestReadDisplayNames runs against the real file where every experiment already has a case, so displayNames[key] is true for all of them and the false path never fires. TestRenderFailsOnMissingDisplayName hands render a hardcoded nil map, bypassing readDisplayNames entirely. If a refactor made readDisplayNames too lenient (mark every experiment constant true regardless of the switch), both existing tests would still pass and the fail-closed guarantee would silently die, resurrecting CRF-1 with a green board. Give readDisplayNames the same fixture treatment readDescriptions gets: two constants, one with a case and one without, assert the first is present and the second absent.

🤖

P3 [CRF-18] TestRenderFailsOnMissingDescription stays green even if the missing-description guard is deleted, so it does not isolate the behavior it names. (Chopper)

The test passes descriptions={"unrelated":"x"} and displayNames=nil over the full ExperimentsKnown list, then asserts only err != nil. With a nil display-name map, every known key also fails the display-name guard. So if a refactor dropped or weakened the description guard, execution would fall through to the display-name guard, err would still be non-nil, and the test named "fails on missing description" would still pass while no longer testing missing descriptions. Give it a valid displayNames map so only the description guard can fire, and assert strings.Contains(err.Error(), "description"). The sibling TestRenderFailsOnMissingDisplayName is correctly isolated but also asserts only err != nil; add strings.Contains(err.Error(), "display name") so the two failure-path tests cannot satisfy each other's assertion.

🤖

P3 [CRF-20] The page's completeness rests on the hand-maintained ExperimentsKnown list, which the generator parses enough to verify but never does. (Meruem, raised as Note; orchestrator P3)

The page promises every experiment a deployment can enable, and render derives that set from known (codersdk.ExperimentsKnown), whose only completeness guarantee is the comment "ExperimentsKnown should include all experiments defined above." No test or lint enforces it. Failure mode: a developer adds an Experiment constant with a DisplayName case and a comment but forgets to append it to ExperimentsKnown. Generation still succeeds (it only iterates known), so the page silently omits the experiment. I verified this is reachable, not merely cosmetic: in coderd/coderd.go ReadExperiments logs "ignoring unknown experiment" for a name not in ExperimentsKnown but then appends it unconditionally, so the omitted experiment is still user-enableable via --experiments=<key>. That is exactly the drift this generator advertises it prevents. Rated P3 rather than Note because it defeats the headline "cannot drift" guarantee for a reachable surface; latent only because the current tree has no such gap. Nearly-free fix: readDisplayNames already builds the full set of declared Experiment values; return it (or a sibling) and have render fail when a declared constant is absent from known, so completeness falls out of the parse instead of resting on a comment. (Distinct from CRF-8/CRF-16, which are about how metadata is sourced.)

🤖

Nit [CRF-21] readDisplayNames returns a presence map whose value is always true, so its name and its render parameter mismodel what the value is. (Mafuuu, Gon, Leorio, Zoro, Knov)

The function returns map[string]bool keyed by experiment value, true when the constant has an explicit case in DisplayName; it carries no display-name strings. The real name comes from exp.DisplayName() at render, which uses this map only as a presence check (if !displayNames[key]). A reader meeting it beside descriptions map[string]string in the render signature reasonably expects displayNames[key] to yield the name and instead gets a bool. This is the same class as the CRF-12 rename (commentText -> commentSentence) the author just fixed one function over. Rename to experimentsWithDisplayName / explicitDisplayNames, use map[string]struct{} since the bool value is never false, and add the one-line doc its sibling readDescriptions already carries.

🤖

Note [CRF-16] readDisplayNames re-parses the same source file and re-walks its Experiment ValueSpecs, duplicating the constant extraction readDescriptions already does. (Netero)

Both functions call parser.ParseFile on *source and iterate ast.Preorder to map Experiment constants (readDescriptions builds value->comment, readDisplayNames builds name->value). The extraction predicate is repeated. Consequence is low: the generator runs once at make gen and a divergent constant form fails loudly rather than emitting wrong output. Parsing once and passing the *ast.File to both extractors would remove the duplicated walk; the clean removal is the DOCS-907 structural change, so this is noted for maintenance, not blocking.

🤖