Skip to content

[Bug]: CloudRift provisioning fails with 400 because the API rejects pinned version 2025-05-29 #4379

Closed

Description

@apedintensor

Steps to reproduce

  1. Configure the cloudrift backend with a valid API key (offers are listed correctly, so credentials and instance-types/list work).
  2. Apply a fleet and a task that should provision on CloudRift:
type: fleet
name: cloudrift-5090
nodes: 0..1
resources:
  gpu: RTX5090:1
backends: [cloudrift]
spot_policy: on-demand
max_price: 0.7
idle_duration: 0s
type: task
name: smoke-cloudrift
fleets: [cloudrift-5090]
resources:
  gpu: RTX5090:1
max_price: 0.7
commands:
  - nvidia-smi

Actual behaviour

dstack offer --gpu RTX5090 shows CloudRift offers (e.g. rtx59-7-24-800-ig.1 in ap_northeast_kr_se_1, $0.65), but every provisioning attempt fails immediately and the run ends as no capacity:

Failed to provision in fleet 'cloudrift-5090': tried 1 of 1 offers, all failed.
Errors: rtx59-7-24-800-ig.1 in cloudrift/ap_northeast_kr_se_1: 400 Client Error:
Bad Request for url: https://api.cloudrift.ai/api/v1/recipes/list.

Expected behaviour

The instance is provisioned.

dstack version

0.22.3 (checked that 0.22.4 still pins the same CLOUDRIFT_API_VERSION = "2025-05-29")

Server logs

ERROR dstack._internal.server.background.pipeline_tasks.jobs_submitted: job(...)smoke-cloudrift-0-0: got exception when ...
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.cloudrift.ai/api/v1/recipes/list

Additional information

The CloudRift API now rejects the version that dstack/_internal/core/backends/cloudrift/api_client.py sends in the request envelope ({"version": "2025-05-29", "data": ...}). I reproduced this outside dstack with direct requests (2026-10-17):

Endpoint version sent Result
instance-types/list 2025-05-29 200
auth/me 2025-05-29 200
recipes/list 2025-05-29 400 Malformed recipes/list request: unsupported version: 2025-05-29
recipes/list 2025-01-29, 2025-06-01, 2025-10-01, 2026-01-01 400 unsupported version
recipes/list ~upcoming 200 (response envelope reports version: 2025-01-29)
instances/rent 2025-05-29 400 Json deserialize error: unsupported version: 2025-05-29
instances/rent ~upcoming (deliberately invalid instance type) 400 Invalid instance variant ..., should be in the format <instance_type>.<variant_name>, i.e. the version is accepted
instances/list with {"selector": {"ByStatus": ["Initializing", "Active", "Deactivating"]}} 2025-05-29 400 invalid type: string "Initializing", expected a sequence

As a local experiment, I forced version = "~upcoming" for recipes/list only. Provisioning then got one step further and failed on instances/rent with the same 400 / unsupported version. The instances/list selector also looks like it changed shape. So the client probably needs to move to the current CloudRift API version and request schema, not a one-line fix. I did not rent any instance during these checks.

Other backends (vastai, runpod) on the same server work fine.

cc @6erun @Slonegg in case the CloudRift API change was intentional and there is a recommended version string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions