Skip to content

Add ctx.reschedule for poll-until-done pattern#5

Merged
cgenuity merged 5 commits into
mainfrom
feat/reschedule
May 2, 2026
Merged

Add ctx.reschedule for poll-until-done pattern#5
cgenuity merged 5 commits into
mainfrom
feat/reschedule

Conversation

@cgenuity

@cgenuity cgenuity commented May 2, 2026

Copy link
Copy Markdown
Member

Summary

Adds ctx.reschedule({ delay, at }) for handler-initiated continuation — the poll-until-done pattern (Replicate predictions, OpenAI batch jobs, Mux transcodes, etc.). Inside a handler, calling reschedule(...) records intent for after the run; on successful return the row transitions running → pending with the new scheduledFor instead of running → completed.

dk.handle("check-replicate", async ({ key, reschedule }) => {
  const p = await replicate.predictions.get(key);
  if (terminal(p.status)) return;
  reschedule({ delay: "2m" });
});

What changed

  • Store.rescheduleJob(id, version, scheduledFor) — new store primitive backing the API. running → pending CAS, attempt resets to 0, clears prior-run error state and scheduler_ref, bumps version so stale wakes lose CAS. Memory + Postgres + SQLite + Postgres concurrency tests.
  • ctx.reschedule({ delay | at }) — synchronous void method on HandlerContext. Last call wins; throwing discards intent. Validates the same way as dk.schedule (parsable duration, valid Date, capped at 10 years). Currently scoped to kind="once" jobs — pattern handlers throw (their requeue happens via wait/maxWait). New handler_rescheduled execution result + result-handler branch.
  • job:rescheduled event — fires per rescheduled cycle with scheduledFor and durationMs. Distinct from job:scheduled so observers counting fresh schedules don't overcount poll cycles.
  • dk.schedule returns a discriminated union{ created: true; job } or { created: false; job; skippedReason } where skippedReason is "pending", "running", or "race_lost". Existing destructuring ({ job, created }) keeps working. The new field nudges callers toward ctx.reschedule when they hit the "running" case (the silent self-rescheduling footgun). Exported as ScheduleResult and SkippedReason.

Why

The website's poll-until-done pattern showed dk.schedule(...) being called from inside a handler, but dk.schedule is idempotent — it skips when an active row exists, including the row currently being executed by the handler. So the pattern was silently broken. Adding ctx.reschedule is the right primitive (the row is being rescheduled, not duplicated), and the discriminated-union return on dk.schedule surfaces the footgun in the moment.

Test plan

  • npm run typecheck
  • npm test — 604 passed (16 new in test/reschedule.test.ts, 4 new in test/delaykit.test.ts)
  • npm run test:postgres — 156 passed (1 new concurrency test, 4 new contract tests)
  • Smoke test the example flow manually against a Posthook deployment
  • Update the website's poll-until-done pattern in delaykit-website/lib/patterns.ts:582 to use ctx.reschedule({ delay: "2m" }) after this lands in npm

Notes

  • Pre-existing latent bug: Store.retryJob has the same shape as rescheduleJob (running → pending CAS) but does not bump version. By the same argument used here for rescheduleJob (stale wakes from the prior run cycle should lose CAS), retryJob arguably should bump too. Not changed in this PR — separate concern.
  • Website example fix is intentionally left to a follow-up PR on the website repo so the npm release lands first.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Handlers can now reschedule jobs mid-execution via ctx.reschedule({ delay, at })
    • Enhanced schedule() method returns reason when duplicate jobs are skipped ("pending", "running", or "race_lost")
    • New job:rescheduled event for observability when handlers trigger reschedules
  • Documentation

    • Added "Poll-until-done" pattern guide for managing long-running async workflows

cgenuity added 3 commits May 1, 2026 20:43
- New `Store.rescheduleJob(id, version, scheduledFor)` mirrors
  `retryJob`'s `running → pending` shape but resets `attempt` to 0,
  clears `last_error` / `failure_reason`, and clears `scheduler_ref`
  so the result handler can materialize a fresh wake at the new
  `scheduledFor`. Also clears the missing-handler horizon state
  (`defer_attempts`, `deferred_since`) since this protocol is
  distinct from the failure-mode defer.
- Bumps `version` to introduce a new delivery identity. Stale wakes
  or held snapshots from the prior run cycle (e.g., an in-flight
  Posthook delivery whose hookId predates the reschedule) lose CAS
  rather than acting on the rescheduled row.
- Returns the updated `Job` on success or `null` on CAS loss
  (concurrent `markCompleted` / `markFailed` / `cancelJob` /
  `replaceJob`).
- Memory, Postgres, and SQLite implementations.
- Backend testsuite covers field mutations, version bump, stale-CAS
  loss after reschedule, CAS loss on wrong version, CAS loss on
  non-running status, and clearing of prior-attempt error state.
- Postgres-concurrency test races `rescheduleJob` against
  `markCompleted` over 50 trials — exactly one wins, the row
  satisfies invariants either way.
- New `ctx.reschedule({ delay, at })` lets a handler indicate "not
  done yet, come back at T" — the row transitions `running → pending`
  with the new `scheduledFor` instead of `running → completed` after
  the handler returns successfully. Backs the poll-until-done idiom
  for Replicate / OpenAI batch / Mux-style polling without rotating
  keys.
- Synchronous void method on `HandlerContext`. Last call within a
  run wins. Throwing from the handler discards the intent and falls
  through to normal retry/failure logic.
- `attempt` resets to `0` on each rescheduled delivery — the
  just-finished run is a checkpoint, not a consumed retry.
- Currently scoped to `kind: "once"` jobs. Calling from a debounce
  or throttle handler throws synchronously; pattern handlers requeue
  automatically via their wait/maxWait window.
- Validates exactly one of `delay` or `at`, with the same constraints
  as `dk.schedule` (parsable duration string, valid Date, capped at
  10 years in the future).
- New `JobRescheduledEvent` (`job:rescheduled`) carries the
  post-reschedule `job`, the new `scheduledFor`, and `durationMs`
  for the just-finished run. Distinct from `job:scheduled` so
  observers counting fresh schedules don't overcount poll cycles.
- New `handler_rescheduled` `ExecutionResult` variant; result-handler
  routes through `Store.rescheduleJob` and materializes a fresh wake.
  CAS loss (concurrent cancel/replace/etc.) is a silent no-op.
- `dk.schedule(...)` now returns `{ created: true; job }` or
  `{ created: false; job; skippedReason }` where `skippedReason` is
  `"pending"`, `"running"`, or `"race_lost"`. Existing destructuring
  keeps working — both arms have `job` and `created` — so callers
  that don't care about the discriminator are unaffected.
- The new field surfaces the silent self-rescheduling footgun: a
  developer who calls `dk.schedule("x", { key })` from inside the
  handler for `("x", key)` now sees `skippedReason: "running"`
  instead of an unexplained `created: false`. JSDoc on the variant
  points at `ctx.reschedule({ delay, at })` as the right tool for
  that case.
- `dk.schedule` JSDoc spells out the idempotence contract and the
  ctx.reschedule alternative.
- Exported `ScheduleResult` and `SkippedReason` types.
- Tests cover each `skippedReason` case (`"pending"` from default
  skip, `"running"` from both `skip` and `replace`, `"race_lost"`
  from a forced `replaceJob` CAS loss).
@coderabbitai

coderabbitai Bot commented May 2, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

A handler-initiated job rescheduling feature is added, enabling ctx.reschedule({ delay, at }) to transition a running job back to pending with a new scheduled time. The implementation introduces new types, store methods, executor validation, and event emissions to support poll-until-done patterns scoped to kind="once" handlers.

Changes

Handler-Initiated Job Rescheduling

Layer / File(s) Summary
Type Definitions
src/types.ts
Added SkippedReason, ScheduleResult discriminated union, RescheduleOptions, RescheduleIntent, JobRescheduledEvent, extended HandlerContext.reschedule() method, and Store.rescheduleJob() contract.
Store Contract & Implementations
src/stores/memory.ts, src/stores/postgres.ts, src/stores/sqlite.ts
Each store adds rescheduleJob(id, version, scheduledFor) method that transitions runningpending, increments version, resets attempt/execution fields, clears error/failure state, and returns updated job or null on version mismatch.
Executor Validation & Context
src/executor.ts
Executor provides ctx.reschedule callback, adds resolveRescheduleAt validator (enforces exactly one of delay|at, validates future bounds, parses durations), records reschedule intent, and returns new "handler_rescheduled" status when intent is set.
Schedule Result Discriminator
src/delaykit.ts
DelayKit.schedule return type updated to ScheduleResult with skippedReason field ("pending", "running", or "race_lost") to differentiate duplicate/skip outcomes.
Result Handling & Events
src/result-handler.ts
Recognizes handler_rescheduled status, calls store.rescheduleJob(), emits job:rescheduled event with scheduledFor and durationMs, materializes wake at new schedule.
Exports
src/index.ts
Re-exports new types: ScheduleResult, SkippedReason, RescheduleOptions, JobRescheduledEvent.
Documentation
CHANGELOG.md, README.md, docs/INVARIANTS.md, docs/OPERATIONS.md
Documents new ctx.reschedule handler API, state-machine transition (running → pending with attempt reset and version bump), observability event, skip semantics, restrictions to kind="once", and poll-until-done usage pattern.
Tests
test/reschedule.test.ts, test/delaykit.test.ts, test/postgres-concurrency.test.ts, test/store-contract.ts
Comprehensive test coverage: reschedule state transitions, event emission, validation of options (delay/at exclusivity, future bounds, invalid Date), "last reschedule wins" semantics, throw/discard behavior, pattern handler rejection, schedule result discriminators, CAS race conditions, and store contract compliance.

Sequence Diagram

sequenceDiagram
    participant Handler
    participant Executor
    participant Store
    participant ResultHandler
    participant Scheduler

    Handler->>Executor: handler function executing
    Handler->>Executor: ctx.reschedule({ delay: "1h" })
    Executor->>Executor: resolveRescheduleAt (validate options)
    Executor->>Executor: record rescheduleIntent
    Handler->>Handler: continues execution
    Handler-->>Executor: returns (no throw)
    Executor->>Executor: detect rescheduleIntent set
    Executor-->>ResultHandler: ExecutionResult { status: "handler_rescheduled" }
    ResultHandler->>Store: rescheduleJob(id, version, scheduledFor)
    Store->>Store: running → pending, version++, attempt=0
    Store-->>ResultHandler: updated Job
    ResultHandler->>ResultHandler: emit job:rescheduled event
    ResultHandler->>Scheduler: materializeWake (new pending schedule)
    Scheduler->>Scheduler: schedule next poll at new time
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A hop through the handler's domain, where reschedule paints new delays!
From pending to running, then back once again,
Poll-until-done now flows without pains—
Each rescheduled beat marks the time to retry,
And the once-kind jobs reach their heavens on high. 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The pull request title 'Add ctx.reschedule for poll-until-done pattern' accurately summarizes the main change: introducing a new ctx.reschedule handler API enabling the poll-until-done pattern for rescheduling jobs mid-execution.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/reschedule

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/INVARIANTS.md`:
- Line 169: Update the docs to include the missing rescheduleJob entry: add
rescheduleJob to the CAS-guarded transition list in Invariant 7 alongside
markCompleted, retryJob, etc. (mention that it uses WHERE status = 'running' AND
version = ${version}). Also add rescheduleJob to the method list in Section 9
DB-first flow alongside rescheduleDueAt() and requeueForNextWindow(), noting it
follows the same row-first-then-wake pattern.

In `@src/executor.ts`:
- Around line 206-207: The delay-based reschedule currently returns new
Date(Date.now() + parseDuration(options.delay as string)) without enforcing the
SCHEDULE_MAX_FUTURE_MS ceiling; update the reschedule logic (the branch that
reads options.delay and calls parseDuration) to clamp the computed future
timestamp to Date.now() + SCHEDULE_MAX_FUTURE_MS (or throw if it exceeds) so it
applies the same 10-year max as the `at` path; use the existing
SCHEDULE_MAX_FUTURE_MS constant and keep parseDuration(options.delay) and the
returned Date semantics intact while enforcing the bound.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0600a064-e4f6-478e-81a8-2562db3594a5

📥 Commits

Reviewing files that changed from the base of the PR and between cb65006 and 625c879.

📒 Files selected for processing (16)
  • CHANGELOG.md
  • README.md
  • docs/INVARIANTS.md
  • docs/OPERATIONS.md
  • src/delaykit.ts
  • src/executor.ts
  • src/index.ts
  • src/result-handler.ts
  • src/stores/memory.ts
  • src/stores/postgres.ts
  • src/stores/sqlite.ts
  • src/types.ts
  • test/delaykit.test.ts
  • test/postgres-concurrency.test.ts
  • test/reschedule.test.ts
  • test/store-contract.ts

Comment thread docs/INVARIANTS.md
Comment thread src/executor.ts Outdated
cgenuity added 2 commits May 1, 2026 21:22
- Add `noteMissingHandler` and `rescheduleJob` to the CAS-guarded
  transitions list in INVARIANTS Invariant 7. Add `rescheduleJob` to
  the DB-first-flow method list in Section 9.
- Apply the same `SCHEDULE_MAX_FUTURE_MS` ceiling to
  `ctx.reschedule({ delay })` that the `at` path already enforces.
  A delay greater than 10 years now throws synchronously.
Self-contained smoke test for `ctx.reschedule` against a real
PosthookScheduler. Wires a single "check-job" handler that polls a
fake external API; the API returns "pending" twice, then "succeeded".
Each pending response calls `ctx.reschedule({ delay: "10s" })`; the
succeeded response returns normally and the row marks completed.

Uses Node's built-in `http` module (no Express dep) and converts
between IncomingMessage and Web Request/Response so dk.createHandler()
can verify Posthook signatures and route to the registered handler.

Run instructions, env vars, and Posthook-project setup notes are in
the file header. Linked from docs/OPERATIONS.md "Poll-until-done".

Verified end-to-end against a real Posthook test project: three
signed deliveries, two reschedule cycles, terminal completion.
@cgenuity
cgenuity merged commit cdaa8ef into main May 2, 2026
7 checks passed
@cgenuity
cgenuity deleted the feat/reschedule branch May 2, 2026 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant