Repository navigation
[Bug]: Errors in server logs while deploying a service with a type: sglang router #4300
Description
Steps to reproduce
Create a service (the same configuration as in #4240 issue):
type: service
name: qwen38-27b-agentic-smg
groups:
- replicas: 1
python: "3.12"
commands:
- pip install smg
- smg launch --host 0.0.0.0 --port 8000 --enable-igw --policy cache_aware --model-path Qwen/Qwen3.8-27B
router:
type: sglang
resources:
gpu: 0
- replicas: 1
image: vllm/vllm-openai:qwen38
commands:
- pip install smg-grpc-servicer
- python3 -m vllm.entrypoints.grpc_server --host 0.0.0.0 --port 8000 --model Qwen/Qwen3.8-27B-FP8
resources:
gpu: H100:80GB
port: 8000
model:
type: chat
name: Qwen/Qwen3.8-27B
format: openai
prefix: /v1Actual behaviour
There are two errors in the server logs:
Parent instance <RunModel at 0x78af662b9e40> is not bound to a Session; deferred load operation of attribute 'run_name' cannot proceed– occurs onceCould not fetch server_info for worker http://10.0.0.3:8000: ReadError('')– repeats until the worker is ready
Expected behaviour
No response
dstack version
0.22.1
Server logs
DEBUG dstack._internal.server.background.pipeline_tasks.base:357 Processing service_router_worker_sync item
648bb027-2a5c-447e-80c7-a8e0b391bc37
ERROR dstack._internal.server.background.pipeline_tasks.base:361 Unexpected exception when processing item
Traceback (most recent call last):
File "/home/def/dev/dstack/src/dstack/_internal/server/background/pipeline_tasks/base.py", line 359, in start
await self.process(item)
File "/home/def/dev/dstack/src/dstack/_internal/server/utils/tracing.py", line 26, in wrapper
result = await f(*args, **kwargs)
File "/home/def/dev/dstack/src/dstack/_internal/server/background/pipeline_tasks/service_router_worker_sync.py", line
269, in process
await sync_router_workers_for_run_model(run_for_sync)
File "/home/def/dev/dstack/src/dstack/_internal/server/services/runs/router_worker_sync.py", line 643, in
sync_router_workers_for_run_model
fmt(run_model),
File "/home/def/dev/dstack/src/dstack/_internal/server/services/logging.py", line 26, in fmt
return fmt_entity("run", model.id, model.run_name)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 569, in __get__
return self.impl.get(state, dict_) # type: ignore
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 1096, in get
value = self._fire_loader_callables(state, key, passive)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 1129, in
_fire_loader_callables
return callable_(state, passive)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py", line 585, in __call__
return strategy._load_for_state(state, passive)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/sqlalchemy/orm/strategies.py", line 543, in
_load_for_state
raise orm_exc.DetachedInstanceError(
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <RunModel at 0x78af662b9e40> is not bound to a Session; deferred
load operation of attribute 'run_name' cannot proceed (Background on this error at: https://sqlalche.me/e/20/bhk3)
DEBUG dstack._internal.server.background.pipeline_tasks.base:364 Processed service_router_worker_sync item
648bb027-2a5c-447e-80c7-a8e0b391bc37 in 0.022
----------------------------------
ERROR dstack._internal.server.services.runs.router_worker_sync:421 Could not fetch server_info for worker http://10.0.0.3:8000:
ReadError('')
Traceback (most recent call last):
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 101, in
map_httpcore_exceptions
yield
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 394, in
handle_async_request
resp = await self._pool.handle_async_request(req)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 256, in
handle_async_request
raise exc from None
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 236, in
handle_async_request
response = await connection.handle_async_request(
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/connection.py", line 103, in
handle_async_request
return await self._connection.handle_async_request(request)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 136, in
handle_async_request
raise exc
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 106, in
handle_async_request
) = await self._receive_response_headers(**kwargs)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 177, in
_receive_response_headers
event = await self._receive_event(timeout=timeout)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_async/http11.py", line 217, in _receive_event
data = await self._network_stream.read(
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 32, in read
with map_exceptions(exc_map):
File "/home/def/.local/share/uv/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/contextlib.py", line 153, in
__exit__
self.gen.throw(typ, value, traceback)
File "/home/def/dev/dstack/.venv/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ReadErrorAdditional information
No response
Activity
[-][Bug]: Errors in server logs while deploying a service with gRPC worker[/-][+][Bug]: Errors in server logs while deploying a service with gRPC workers[/+][-][Bug]: Errors in server logs while deploying a service with gRPC workers[/-][+][Bug]: Errors in server logs while deploying a service with a router[/+][-][Bug]: Errors in server logs while deploying a service with a router[/-][+][Bug]: Errors in server logs while deploying a service with a `type: sglang` router [/+]Fix errors in service router worker sync (dstackai#4305)