Skip to content

feat(coderd/tracing): correlate request logs and spans by client_session_id - #27671

Merged
aqandrew merged 16 commits into
mainfrom
devex-659-session-id-tracing-middleware
Aug 19, 2026
Merged

feat(coderd/tracing): correlate request logs and spans by client_session_id#27671
aqandrew merged 16 commits into
mainfrom
devex-659-session-id-tracing-middleware

Conversation

@aqandrew

@aqandrew aqandrew commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What

Adds client_session_id correlation to coderd's HTTP request handling, per the
Connection log collection and correlation RFC.

Clients attach a per-session correlation ID to every API request via W3C
baggage using the client_session_id key. This change makes coderd's tracing
middleware read that baggage member and:

  • add client_session_id to the per-request log context so all logs for a
    request (and the handlers it calls) can be correlated by a single ID, and
  • set client_session_id as a span attribute when tracing is enabled.

Per RFC requirement 6.1, the value is added to the log context even when
tracing is disabled
(the middleware previously returned early when no tracer
provider was configured, so baggage was never read). The client_session_id is
validated as a 32-character hexadecimal string (a 16-byte value, per RFC
requirement 1) to guard against logging arbitrary client-controlled baggage
values.

Scope

This is DEVEX-659 and is intentionally limited to the coderd tracing
middleware. It is the first piece of a stack: the web terminal client change
(DEVEX-663) that generates and sends the client_session_id will be stacked on top
of this PR. No client currently sends client_session_id baggage, so this change is
a no-op until the client work lands.

Testing

  • coderd/tracing: new unit tests cover validSessionID, baggage extraction
    (sessionIDFromHeaders), and the middleware end to end, asserting
    client_session_id lands on the log context with tracing enabled and disabled,
    is exposed as a span attribute when tracing is enabled, and that
    absent/malformed baggage is ignored.
  • Existing Test_Middleware route-matching behavior is unchanged.
Design notes / decision log
  • Where the value is read: the existing tracing.Middleware runs high in
    the coderd middleware stack (coderd/coderd.go), before request-id and
    request-logger middleware, and already matches the /api, /api/**, app
    proxy, and external-auth routes. Reading baggage here means the client_session_id
    is on the context before the request logger and handlers run, so it flows
    into all downstream slog calls that use the request context. This mirrors
    the existing request_id pattern in httpmw.AttachRequestID
    (slog.With(ctx, ...) + span attribute).
  • Works when tracing is off: the middleware now gates only on the route
    matcher, extracts baggage and adds client_session_id to the log context for all
    matched routes, and only then branches on whether a tracer is configured.
    When a tracer is present, client_session_id is additionally set as a span
    attribute.
  • Explicit baggage propagator: extraction uses propagation.Baggage{}
    directly rather than the global text map propagator, so it does not depend
    on the global propagator being configured (also makes it deterministic in
    tests).
  • Validation: only a 32-char hex string is accepted (lower or upper case).
    Malformed values are dropped rather than logged, preventing log/attribute
    pollution from arbitrary client-supplied baggage.
  • Out of scope for this PR (tracked elsewhere): client generation/sending
    of client_session_id (DEVEX-663, web terminal), the equivalent agent-side
    middleware (RFC 6.2), connection_logs.client_session_id (RFC 12), and additional
    connection state-change logging (RFC 7-13).

Opened by Coder Agents on behalf of @aqandrew.

@linear-code

linear-code Bot commented Jul 30, 2026

Copy link
Copy Markdown

DEVEX-659

Copy link
Copy Markdown
Contributor Author

Comment thread coderd/tracing/httpmw_internal_test.go

@jeremyruppel jeremyruppel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, tightly-scoped change. The strict 32-char hex validation at the trust boundary and the explicit (non-global) baggage propagator are both the right calls, and the tests exercise real behavior through a slog sink and a recording span rather than tracing internals.

A few things to look at: 1 P2 and 2 P3 findings, 1 nit, and 4 observations across 9 inline comments. Nothing blocking.

Review generated by Coder Agents (deep-review) on behalf of @jeremyruppel.

Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw.go Outdated
Comment thread coderd/tracing/httpmw.go Outdated
Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw_test.go
Comment thread coderd/tracing/httpmw_test.go
Comment thread coderd/tracing/httpmw_test.go Outdated
aqandrew added a commit that referenced this pull request Aug 11, 2026
- adds a `generateConnectionSessionId` helper function which reuses our
existing `generateRandomString` logic
- adds tests for `generateConnectionSessionId` in utils/random.test.ts
(utils/random.ts was previously untested)
- renames our existing `generateRandomString` function to be
`generateRandomBase64String`

additional context in
#27671 (comment)

This PR doesn't change any of our UIs. The new
`generateConnectionSessionId` function is a piece of frontend plumbing
related to DEVEX-663. I originally implemented #27677 so that web
terminal connections to workspaces would be identified by uuids--but
according to the RFC for DEVEX-663, those connection session ids should
be lowercase hexadecimal strings (not uuids).
@aqandrew
aqandrew requested a review from jeremyruppel August 11, 2026 19:12
Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw_test.go Outdated
Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw.go
@aqandrew aqandrew changed the title feat(coderd/tracing): correlate request logs and spans by session_id feat(coderd/tracing): correlate request logs and spans by client_session_id Aug 12, 2026
Read the session_id baggage member on API requests and attach it to the
per-request log context and, when tracing is enabled, as a span attribute.
The value is added to the log context even when tracing is disabled so logs
can always be correlated by session_id, per the connection-log RFC.

The session_id is validated as a 32-character hexadecimal string to guard
against logging arbitrary client-controlled baggage values.

Part of DEVEX-659.
…ge key

Reference the exported SessionIDBaggageKey constant when reading the baggage
member and when constructing baggage headers in tests, instead of repeating
the string literal. The emitted slog field and span attribute names remain
snake_case string literals ("session_id"), as required by the slog field-name
lint rule.
The connection-log RFC was updated to mandate lowercase hexadecimal for
session IDs so that case-sensitive searches correlate reliably. Tighten
validSessionID to reject uppercase and update the tests accordingly.
Add a test asserting the baggage key, log field name, and span attribute
name all equal "session_id". slog field names must be snake_case string
literals, so the log field and span attribute cannot reference the
SessionIDBaggageKey const directly; this test guards against them drifting
apart and silently breaking log/trace correlation (PR review P2).
…utes

The middleware extracts session_id only on matched API/app routes. Add a
subtest that sends well-formed baggage to a non-matching path (/index.html)
and asserts session_id is absent from both the log fields and the span, so a
regression that logged client-controlled baggage on every request would be
caught (PR review P3).
The negative span assertions checked only that a specific value was not set,
so a bug setting session_id to a different derived value would pass. Scan the
span attributes for any key equal to session_id and assert its absence in the
malformed-baggage and non-matching-route cases (PR review P3).
Use hex.DecodeString to check the value is 16 bytes of hex, matching the
codebase convention, then require hex.EncodeToString(b) == s so only the
canonical lowercase encoding is accepted (hex.DecodeString also accepts
upper-case) (PR review nit).
Fold recordingTracer into fakeTracer via an optional recording span, and
add a tracing-enabled + no-baggage subtest that pins the branch which must
not set an empty session_id span attribute or log field.
@aqandrew
aqandrew force-pushed the devex-659-session-id-tracing-middleware branch from 17508c1 to 3bac44f Compare August 13, 2026 00:07
The baggage is well-formed; it is the session ID value that is malformed.
Addresses review feedback on #27671.
The tracer provider is never nil in production (coderd defaults it to a
no-op provider when tracing is disabled), so drop the tracer == nil
special case. Default a nil provider to no-op and always start the span
and set the client_session_id attribute; the no-op tracer discards the
span. Addresses review feedback on #27671.
@aqandrew
aqandrew requested a review from code-asher August 17, 2026 21:59
Browser WebSocket clients such as the web terminal PTY cannot set
arbitrary baggage headers. Extend the tracing middleware to read the
client_session_id from baggage first, then fall back to the
client_session_id query parameter. The query value passes the same
lowercase 32-hex validation, and baggage takes precedence when both are
present. This lands the session ID on both the log context and the
trace span for PTY requests.
Comment thread coderd/tracing/httpmw.go
Comment thread coderd/tracing/httpmw_test.go Outdated
Each subtest builds its own fakeTracer and recordingSpan, and the
middleware handles each request synchronously on a single goroutine.
The span's attributes are read only after ServeHTTP returns on the same
goroutine, so there is a strict happens-before ordering and no shared
access. The sync.Mutex and defensive slices.Clone are therefore
unnecessary. Verified with go test -race.
…tracing-middleware

# Conflicts:
#	coderd/tracing/httpmw_test.go
Extract a sessionIDFromQueryString helper that mirrors
sessionIDFromHeaders: extract the client_session_id from its source,
validate it, and return an empty string when absent or malformed.
sessionIDFromRequest now composes the two extractors, keeping baggage
precedence. Each extractor stays self-validating so sessionIDFromHeaders
keeps its validated contract, which the agent SessionIDMiddleware relies
on. Adds a symmetric TestSessionIDFromQueryString internal test.
@aqandrew
aqandrew requested a review from jeremyruppel August 18, 2026 23:28
@aqandrew
aqandrew merged commit 71e95a3 into main Aug 19, 2026
29 checks passed
@aqandrew
aqandrew deleted the devex-659-session-id-tracing-middleware branch August 19, 2026 00:28
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 19, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants