Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: utopia-php/queue
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: utopia-php/queue
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: feat/queue-keda
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 10 commits
  • 14 files changed
  • 2 contributors

Commits on Jul 14, 2026

  1. feat(queue): KEDA ScaledJob POC — payload stays in Redis 

    Alternative to the env-var KubernetesJob broker: instead of inlining the payload
in a per-message Job, producers enqueue to the ordinary Redis broker and a KEDA
ScaledJob scales one-shot worker Jobs off the queue depth. Each worker drains
the queue with the same Redis broker and exits — no custom broker, no payload on
any Kubernetes object.

Adds a drain worker, Dockerfile, redis + ScaledJob manifests, a kubectl-driven
e2e (KedaTest) proving KEDA spawns Jobs that drain the queue, and keda-e2e.sh
(kind + KEDA + redis). Verified end-to-end on kind. See servers/Keda/README.md
for the comparison.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    6c94dee View commit details
    Browse the repository at this point in the history

  2. test(queue): run the KEDA e2e in CI via a shared keda-lib.sh 

    Factor the kind + KEDA + Redis + ScaledJob provisioning into tests/keda-lib.sh
(keda_up/keda_down) and have both keda-e2e.sh and the package's e2e.sh use it,
so `bin/monorepo test queue` (and thus CI) stands up KEDA and runs KedaTest
against a real cluster instead of skipping it.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    00ff389 View commit details
    Browse the repository at this point in the history

  3. test(queue): address review on the KEDA POC 

    - keda-lib.sh: pin + SHA-256-verify Helm (same as kind/kubectl) instead of
  curl|bash of an unpinned installer; only tear down a kind cluster this run
  created, never a pre-existing one.
- KedaTest: also assert the .failed.* list is empty — draining the main queue
  alone doesn't prove success since receive() pops before handling; note that
  enqueue() mirrors Redis::enqueue()'s envelope.
- k8s.yaml: correct the scaling comment (KEDA scales N Jobs, workers batch-drain)
  and document the backoffLimit/processing-orphan crash-recovery caveat.
- README: crash-recovery section (processing-list reaper via Publisher::retry).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    97f307f View commit details
    Browse the repository at this point in the history

  4. feat(queue): add Adapter\KubernetesJob (run-to-completion drain adapter) 

    The KEDA approach still needs one bit of library code: a consumer that drains
the queue and exits (so a Job completes) instead of blocking like the Swoole/
Workerman adapters. Extract that out of the test worker into
Utopia\Queue\Adapter\KubernetesJob, cover it with a bare-host unit test
(KubernetesJobAdapterTest), and have the KEDA worker run it via Server. Producers
are unchanged — they still enqueue with any Publisher (e.g. the Redis broker).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    4c0a79d View commit details
    Browse the repository at this point in the history

  5. fix(queue): harden KubernetesJob run-to-completion adapter against po… 

    …d termination and boot failures

Register pcntl SIGTERM/SIGINT handlers in the KubernetesJob consume loop so
graceful pod termination drains the in-flight message instead of being SIGKILLed
(PHP is PID 1 in the Job container, so an unhandled SIGTERM is ignored). Run the
workerStart callbacks under try/finally so workerStop (Timer cleanup) always runs.
Add runsToCompletion() to Adapter, overridden true in KubernetesJob, and rethrow
boot/consume-loop failures from Server::start() for run-to-completion adapters so
the Job fails (honouring backoffLimit) rather than exiting 0. Guard the Redis
broker claim writes after the pop and requeue the payload on failure so a mid-claim
error no longer strands the message.

Co-Authored-By: Claude Fable 5 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    66f25e3 View commit details
    Browse the repository at this point in the history

  6. fix(queue): install pcntl in the KEDA e2e worker image 

    Co-Authored-By: Claude Fable 5 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    6ad818e View commit details
    Browse the repository at this point in the history

  7. refactor(queue): Swoole-native signal handling in KubernetesJob 

    Co-Authored-By: Claude Fable 5 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    81309cd View commit details
    Browse the repository at this point in the history

  8. fix(queue): run the whole KubernetesJob lifecycle inside one scheduler 

    Co-Authored-By: Claude Fable 5 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    52b9c7d View commit details
    Browse the repository at this point in the history

  9. revert(queue): drop requeue-on-failed-claim in the Redis broker 

    Co-Authored-By: Claude Fable 5 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    9a67396 View commit details
    Browse the repository at this point in the history

  10. refactor(queue): fold single-use helpers into their call sites 

    Co-Authored-By: Claude Fable 5 <[email protected]>
    @abnegate @claude
    abnegate and claude committed Jul 14, 2026
    Configuration menu
    Copy the full SHA
    8fa82ed View commit details
    Browse the repository at this point in the history
Loading