Skip to content

Implement backend-specific offers cache #3021

Closed as not planned
Closed as not planned

Description

@r4victor

Currently, offers cache is implemented via Compute.get_offers_cached(requirements), which caches backend offers given requirements. Since cache is not reused for different requirements, it's expensive to get offers for every existing fleet when provisioning a run (#3020). The solution is to make the offers cache backend-specific:

  • We will call get_offers(requirements) for each fleet. Offer caching will not be shared at the get_offers level, but will be an implementation detail of the backend.
  • Backends where offers do not depend on requirements will have a similar implementation: collect all offers and cache them, then apply the common offer filtering logic.
  • Backends where offers depend on requirements will be able to cache not the dstack offers (since offers cannot be formed without requirements), but the API requests from which offers will later be formed using the requirements.

Essentially, the idea is to cache only the expensive operations (API calls) and allow doing fast operations (requirements filtering) many times.

I also suggest we move offers collection and filtering logic from gpuhunt to dstack as a part of this issue.

Activity

  1. self-assigned this
    on Sep 10, 2025
  2. r4victor commented on Oct 9, 2025

    @r4victor
    CollaboratorAuthor

    #3091 implemented requirements-independent offers cache on the dstack side for all backends expect cudo and vastai. Cudo and vastai require caching provider's API calls, which currently can be done in gpuhunt only. Until implemented, requesting cudo and vastai offers will be less efficient. We can postpone the implementation and do it when moving offer collection logic from gpuhunt to dstack.

  3. removed their assignment
    on Oct 9, 2025
  4. github-actions commented on Nov 9, 2025

    @github-actions

    This issue is stale because it has been open for 30 days with no activity.

  5. github-actions commented on Nov 30, 2025

    @github-actions

    This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen the issue if it is still relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions