Skip to content

Reuse the runner's dependency install on Git push deploys [skip ci] - #316

Merged
matuscongrady merged 3 commits into
mainfrom
perf/runner-install-reuse
Oct 23, 2026
Merged

matuscongrady merged 3 commits into
mainfrom
perf/runner-install-reuse

Conversation

@matuscongrady

@matuscongrady matuscongrady commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Two CLI changes that make Git push deploys faster. The runner side is stacktape/console-app#140.

The CLI reuses the runner's dependency install. On a push deploy, the EC2 runner installs the checkout's dependencies. The CLI then ran npm ci (about 9 s on the runner for the benchmark's lambda fixture), because isCompletedNpmInstall refused the tree. The fixture is an npm workspace, npm leaves out optional packages for other platforms (@esbuild/darwin-*), so the hidden lockfile never equals the lockfile, and esbuild and prisma have install scripts. npm does not record whether install scripts ran, so content alone cannot prove such a tree. The runner therefore says so explicitly. The installer skips its install and writes its marker, so a nested Stacktape process in that checkout also skips it, when three things hold. STP_RUNNER_INSTALLED_DEPENDENCIES names the install directory and node_modules exists there. STP_RUNNER_INSTALLED_LOCKFILE names the same lockfile the CLI follows; the runner and the CLI pick lockfiles in different orders. For pnpm, the CLI would also run the pnpm on PATH, not a declared version or the one a v6 lockfile needs. Otherwise it installs as before. The runner sets the variable only after a successful install of a fresh worktree, so a lockfile change still installs. Laptops and other CI pipelines behave as before.

The npm check itself now also accepts workspaces (it verifies the links, each workspace's manifest against its lockfile entry, and the workspace patterns) and missing optional packages whose os, cpu or libc excludes this machine. Any other missing optional package, from --omit=optional or an older cached tree, still installs. That lets a pipeline's own npm ci count, but only for trees without install scripts, which keep refusing as before.

A Python code swap no longer pulls its local base image. The swap image starts FROM stacktape-python-swap-base:<id>, which is the image built just before it and exists only in the local store. The build treated it as a base image to keep pulled, so every Python push ran a docker pull against Docker Hub that failed (0.7 s locally).

Measured with the runner's command sequence on this machine (fresh checkout, the runner's install, CI=true stacktape deploy to the e2e account, alternating before and after, n=5 each):

  • lambda-5 fixture: median from git push-equivalent start to deployed went from 41.8 s to 24.7 s. Initialize went from 15.1 s to 1.6–4.6 s; the remaining spread is AWS API latency. The second install took 12.2–14.5 s locally.
  • container-python fixture, code swap: Build & Package median went from 3.2 s to 2.5 s.
  • pnpm, yarn and bun variants of the lambda fixture deploy with the variable set (n=2–3 each). Their second install was already a 0.03–0.3 s no-op.

Checks: new dependency-installer.spec.ts builds real npm workspace trees offline (links, a package skipped for another platform, changed manifests, a moved link, an incomplete install, and the runner path keeping node_modules untouched). The extended railpack.test.ts asserts that the swap build names no base image. test:src, test:fresh-install, packaging test, typecheck and check:integrated pass, except packaging-harness.spec.ts "measures a shape and reports which artifacts each edit changes", which fails the same way on origin/main.

matuscongrady and others added 3 commits October 23, 2026 22:51
…hon code swap [skip ci]

On a Git push deploy the EC2 runner installs the checkout's dependencies, and
the CLI then ran npm ci again because it could not verify the tree: npm
workspaces, skipped optional platform packages and install scripts all made the
check refuse it. The runner now names the directory it installed in through
STP_RUNNER_INSTALLED_DEPENDENCIES, and the CLI reuses that install for any
package manager. The npm check also accepts workspaces and skipped optional
packages, for pipelines without install scripts.

A Python code swap image starts from the base image built just before it, which
only the local store holds; the build no longer asks Docker Hub for it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…nd pnpm [skip ci]

The runner and the CLI pick lockfiles in different orders, and the runner
always runs its own pnpm. The CLI now keeps the runner's tree only when
STP_RUNNER_INSTALLED_LOCKFILE names the lockfile the CLI follows and, for
pnpm, when the CLI would also run the pnpm on PATH. The npm check accepts a
missing optional package only when its os, cpu or libc excludes this machine,
so a tree restored from an older cache or installed with --omit=optional is
installed again.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant