You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, offers cache is implemented via Compute.get_offers_cached(requirements), which caches backend offers given requirements. Since cache is not reused for different requirements, it's expensive to get offers for every existing fleet when provisioning a run (#3020). The solution is to make the offers cache backend-specific:
We will call get_offers(requirements) for each fleet. Offer caching will not be shared at the get_offers level, but will be an implementation detail of the backend.
Backends where offers do not depend on requirements will have a similar implementation: collect all offers and cache them, then apply the common offer filtering logic.
Backends where offers depend on requirements will be able to cache not the dstack offers (since offers cannot be formed without requirements), but the API requests from which offers will later be formed using the requirements.
Essentially, the idea is to cache only the expensive operations (API calls) and allow doing fast operations (requirements filtering) many times.
I also suggest we move offers collection and filtering logic from gpuhunt to dstack as a part of this issue.
#3091 implemented requirements-independent offers cache on the dstack side for all backends expect cudo and vastai. Cudo and vastai require caching provider's API calls, which currently can be done in gpuhunt only. Until implemented, requesting cudo and vastai offers will be less efficient. We can postpone the implementation and do it when moving offer collection logic from gpuhunt to dstack.
Activity
r4victor commentedon Oct 9, 2025
#3091 implemented requirements-independent offers cache on the dstack side for all backends expect cudo and vastai. Cudo and vastai require caching provider's API calls, which currently can be done in gpuhunt only. Until implemented, requesting cudo and vastai offers will be less efficient. We can postpone the implementation and do it when moving offer collection logic from gpuhunt to dstack.
github-actions commentedon Nov 9, 2025
This issue is stale because it has been open for 30 days with no activity.
github-actions commentedon Nov 30, 2025
This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen the issue if it is still relevant.