feat: correlate coder CLI workspace sessions with client_session_id - #28313
Merged
Merged
Conversation
Generate a 16-byte lowercase-hex client session ID for the ssh sub-command (honoring CODER_TRACE_SESSION_ID when set) and attach it to every request as client_session_id W3C baggage, to the command's log context, and to CLI network telemetry, per the connection-log RFC. - tracing: add NewSessionID and export ValidSessionID. - codersdk/cli: attach client_session_id baggage to all ssh requests via a transport wrapper. - cli: resolve the ID from CODER_TRACE_SESSION_ID or generate one; add it to the ssh log context. - tailnet: add client_session_id to TelemetryEvent and plumb it from the CLI through workspacesdk DialAgent.
aqandrew
changed the title
aqandrew
marked this pull request as ready for review
August 20, 2026 03:31
spikecurtis
reviewed
Aug 20, 2026
Resolve the client session ID once per invocation in a global serpent middleware installed via cmd.Walk, instead of only in coder ssh. The middleware stores the ID on the invocation context both as a slog field (so any log written with the invocation context carries client_session_id regardless of which logger emits it) and under a private key for non-log consumers. createHTTPClient now reads the ID from the invocation context to attach W3C baggage, and ssh reads it from the context to forward as tailnet telemetry. This removes the RootCmd.clientSessionID shared-state field. Based on Spike's review on #28313. Generated by Coder Agents.
Long-running daemon commands (coder server, agent, provisionerd, etc.) should not carry a client session ID: it is meaningless for them and prepending client_session_id to their logs broke log-volume and log-format expectations (TestServer/SpammyLogs, TestServer/Prometheus). Gate clientSessionIDMiddleware on a new annotationClientSessionID opt-in annotation. Commands that establish a client session opt in: coder ssh, start, stop, and update. Daemons never opt in, so their logs, request baggage, and telemetry stay free of a session ID. Generated by Coder Agents.
Bumping the shared tailnet/agent API to 2.12 (for the client_session_id TelemetryEvent field) made agentproto.CurrentVersion 2.12, but postStartup connected via the version-pinned ConnectRPC211WithRole, so TestWorkspaceAgent_Startup/OK recorded APIVersion 2.11 and failed the equality assertion. Connect at proto.CurrentVersion so the recorded version tracks future bumps. Generated by Coder Agents.
Set the client session ID as W3C baggage on the coordinate websocket
handshake headers in DialAgent, mirroring how the session token is
attached. coderd's tracing middleware reads client_session_id from
baggage, so the /api/v2/workspaceagents/{id}/coordinate request is now
logged with the session ID.
Setting it explicitly at the dial site (rather than relying on the CLI's
baggage-injecting transport) makes propagation intentional and works for
non-CLI callers that construct a plain codersdk client. Prefer the header
over a query param since a Go client can set headers and a query param
would spread the ID into request URLs and access logs across hops.
spikecurtis
approved these changes
Aug 27, 2026
aqandrew
changed the title
Agent HTTP API calls go through a separate per-request HTTP client that does not run the CLI's baggage transport, so client_session_id never reached the agent. Attach it to the agent connection's extra headers in DialAgent (when ClientSessionID is set) so every agent API request carries the baggage and the agent's tracing middleware logs it. Extract clientSessionIDBaggage as the shared source of the baggage value for both the coordinate handshake and the agent extra headers, and add a unit test for it.
code-asher
approved these changes
Sep 1, 2026
code-asher
left a comment
code-asher
left a comment
Member
There was a problem hiding this comment.
Looks good to me! Just had some tiny nits about comments and a couple test file names.
…ames correspond with tested modules
aqandrew
enabled auto-merge (squash)
September 2, 2026 19:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements DEVEX-664: the coder CLI commands that open a workspace connection now generate a client session ID and attach it to their requests, logs, and existing network telemetry, per the connection-log RFC.
Commands opt in explicitly with a
client_session_idannotation. The opted-in set iscoder ssh,coder start,coder stop, andcoder update. Long-running daemon commands (coder server,agent,provisionerd, and so on) deliberately stay out, so they never carry a meaningless session ID in their logs, request baggage, or telemetry.CODER_TRACE_SESSION_IDis set (so Toolbox and the VS Code plugin can correlate the CLI they spawn), that value is used verbatim; a warning is logged if it is not the canonical 32-char lowercase hex form, since coderd/agent middleware drop non-canonical values.client_session_idW3C baggage via a transport wrapper, independent of whether tracing is enabled. The already-merged coderd tracing middleware (feat(coderd/tracing): correlate request logs and spans by client_session_id #27671) reads it.workspacesdk.DialAgent, so coderd's/api/v2/workspaceagents/{id}/coordinateendpoint logs it (it does not rely on the CLI transport for that dial).TelemetryEvent(client_session_id, new field 20; proto minor version bumped) and plumbed from the CLI throughworkspacesdk.DialAgent.Scope is the coder CLI client only (RFC glossary + requirement 5), and within the CLI only the opted-in workspace-connecting commands above. This PR is independent of #28039 (the agent middleware that consumes the baggage); it depends only on
tracing.SessionIDBaggageKey, which is already onmainvia #27671.Out of scope
RFC requirements 7-16 (connection-state change logging,
connection_logscolumns, log buffering, routing/disconnect-reason logs, coordination-protocol plumbing,agentssh/reconnectingptycommand logging, support-bundle log capture) and requirements 4.2/4.3 (VS Code telemetry, Prometheus stats) are separate tickets. IDE plugins, Toolbox, and the daemon commands are also out of scope.Testing
go test ./coderd/tracing/... ./cli/... ./tailnet/... ./codersdk/workspacesdk/...(unit tests for the generator, the env/generate resolver, per-command opt-in, the baggage transport wrapper, the coordinate-dial baggage helper, and the telemetry field).go build ./...,go vet, andgolangci-lintpass on the touched packages.make tailnet/proto/tailnet.pb.goregenerated the proto; the proto minor version was bumped andpostStartupnow connects at the current agent API version soTestWorkspaceAgent_Startup/OKtracks it.Implementation plan
Scope mapping (RFC -> DEVEX-664)
In scope (the CLI workspace-connecting subset of RFC phase 2):
client_session_idbaggage, including the coordinate websocket handshake. (The query-param fallback is browser-only; not applicable to the CLI.)CODER_TRACE_SESSION_ID; when set, use it instead of generating.Resolved decisions
client_session_idannotation; opted-in set iscoder ssh,coder start,coder stop,coder update. Daemon commands are excluded.CODER_TRACE_SESSION_ID: when set, use the value verbatim; warn if it is not canonical 32-char lowercase hex. When unset, generate a fresh ID.TelemetryEvent); requires a new proto field + plumbing and a proto minor version bump.DialAgent(header, not query param) so it is logged server-side without relying on the CLI transport.tracing.NewSessionID()and export the validator astracing.ValidSessionIDfor a single source of truth.Implementation
coderd/tracing: addNewSessionID()(16 random bytes, hex-encoded); exportValidSessionID.cli: resolve the ID fromCODER_TRACE_SESSION_IDor generate; attachclient_session_idbaggage to every request viawrapTransportWithSessionIDHeader, gated on theannotationClientSessionIDopt-in; add it to the command log context.tailnet+codersdk/workspacesdk: addclient_session_idtoTelemetryEvent, anOptions.ClientSessionID/Conn.clientSessionIDset innewTelemetryEvent, aDialAgentOptions.ClientSessionIDplumbed from the opted-in commands, and the coordinate-handshake baggage inDialAgent.This PR was created by Coder Agents on behalf of @aqandrew.