Repository navigation
Implement volumes force detach - #2242
Merged
Merged
Conversation
r4victor
marked this pull request as ready for review
January 30, 2025 07:47
un-def
reviewed
Jan 30, 2025
un-def
approved these changes
Jan 30, 2025
pranitnaik43
pushed a commit
to bahaal-tech/dstack
that referenced
this pull request
Feb 9, 2025
* Add comments on termination * Simplify job termination and fix instance locking * Fix tests * Implement volume force detach * Fix duration parameters parsing * Filter our instances with detaching volumes * Add docs on Force detach
pranitnaik43
pushed a commit
to bahaal-tech/dstack
that referenced
this pull request
Mar 4, 2025
* Add comments on termination * Simplify job termination and fix instance locking * Fix tests * Implement volume force detach * Fix duration parameters parsing * Filter our instances with detaching volumes * Add docs on Force detach
pranitnaik43
pushed a commit
to bahaal-tech/dstack
that referenced
this pull request
Mar 5, 2025
* Add comments on termination * Simplify job termination and fix instance locking * Fix tests * Implement volume force detach * Fix duration parameters parsing * Filter our instances with detaching volumes * Add docs on Force detach
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2218
The PR:
process_terminating_jobcould be called fromprocess_terminating_run(e.g. for non-running provisioned jobs) (evidently to safe a processing iteration). Now,process_terminating_runonly marks jobs as TERMINATION, and onlyprocess_termination_jobsbackground task performs the jobs termination. This simplifies locking management significantly and overall logic.stop_durationto run configuration/profile to control max duration a job waits until force terminating.*_durationparameters parsing. E.g."off"was handled formax_durationbut notoffand vice versa foridle_duration.truewas allowed but parsed as 1.Next:
dstack stopCLI command to wait until the run enters the terminal status. Currently, the API is async meaning that run may be terminating for some time after the API returns.