Skip to content

Create placeholder instance models - #3821

Merged
r4victor merged 17 commits into
masterfrom
issue_3665_placeholder_instance_models
Apr 24, 2026
Merged

r4victor merged 17 commits into
masterfrom
issue_3665_placeholder_instance_models

Conversation

@r4victor

@r4victor r4victor commented Apr 23, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #3665
Fixes #3663

  • Refactor JobSubmittedPipeline flow to create a "placeholder" InstanceModel before provisioning the instance in the cloud. This helps with proper fleets' nodes.max accounting and instance_num reservation, and prepares for proper instance termination in case of stale worker (still TBD).
  • Add InstanceModel.provisioning_job_id to distinguish placeholder instances vs pending instances processed by InstancePipeline.
  • Carefully update all the places to account for placeholder instances where fleet instances are filtered for empty check, placement group selection, master election, etc.
  • Disallow deleting "placeholder" instances to avoid racing with pipelines.

@r4victor
r4victor merged commit f4d9513 into master Apr 24, 2026
25 checks passed
@r4victor
r4victor deleted the issue_3665_placeholder_instance_models branch April 24, 2026 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

process_submitted_jobs should create InstanceModel before provisioning instances [Bug]: Can over-allocate instances in fleet past nodes-max when using rest API

1 participant