Skip to content

Don't count requests rejected before routing as service hits - #4387

Merged
jvstme merged 1 commit into
dstackai:masterfrom
vitostrokov:issue_4384_stats_flagless_requests
Oct 21, 2026
Merged

jvstme merged 1 commit into
dstackai:masterfrom
vitostrokov:issue_4384_stats_flagless_requests

Conversation

@vitostrokov

Copy link
Copy Markdown
Contributor

Fixes #4384

Nginx rejects some requests, such as ones with a malformed request line, before the
server-level set $dstack_replica_hit 0 runs. The flag is then logged empty, and the line ends
in a trailing space. line.split() turned such lines into 4 cells, which the collector read as
pre-0.19.11 log lines and counted as hits. A single stray request could then scale a service up
from zero.

The collector now splits on single spaces after stripping the newline. A current-format line
with an empty flag keeps 5 cells, and an empty flag counts as no hit. Legacy 4-field lines are
parsed as before.

Testing:

  • uv run pytest src/tests/_internal/proxy/gateway/: 60 passed.
  • The new test_collect_stats[ignores-requests-rejected-before-routing] case fails without the
    fix.
  • uv run ruff check and uv run ruff format --check on the changed files: clean.

🤖 Generated with Claude Code

Nginx answers some requests, e.g. ones with a malformed request line,
with 400 before the server-level `set $dstack_replica_hit 0` runs, so
the flag is logged empty. The stats collector read these 4-field lines as
pre-0.19.11 logs and counted them as replica hits, so stray internet
traffic could scale a service up from zero.

Tell the two cases apart by the trailing separator that the current log
format leaves when the flag is empty, and treat an empty flag as no hit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@peterschmidt85
peterschmidt85 requested a review from jvstme October 18, 2026 08:14

@jvstme jvstme left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks for reporting and fixing!

@jvstme
jvstme merged commit eba42b9 into dstackai:master Oct 21, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Requests that nginx rejects with 400 count as service hits and scale services up from zero

2 participants