Conversation
1847e74 to
d6805b7
Compare
Codex PR Review
Go checks could not run because the read-only filesystem prevented module-cache creation. |
b5e92f0 to
12e9027
Compare
|
@mrsimonemms I’d like your opinion on the third bullet in Codex’s review. The fixed zigflow-test queue is intentional and documented in this PR’s description. For v1 I see two realistic options, not a third “distributed lock” path for now:
I am not keen to add a third-party dependency (Redis, etc.) purely for cluster-wide locking in the CLI. That feels out of scope for Zigflow as an engine/CLI, even if a distributed lock would work in theory. My bias is option 1 for v1, with docs/CI guidance where needed (e.g. namespace per job), unless you think per-run queues are clearly better. Does that align with how you expect zigflow test to be used? |
12f2451 to
a339452
Compare
|
Actually, I’m now leaning towards the per-run task queue. The part I’m still uneasy about is how to bound concurrency so we don’t spawn one active worker per concurrent zigflow test invocation without an explicit limit. |
dee9f3e to
b636110
Compare
|
Yeah, it's a difficult one. I wonder if this is one of those things where we should open a discussion on Slack and get a wider response from users? Also, I'm at an offsite in the US this week so I'm not going to be as responsive this week - I need to have a bit of a think about your questions. One thing - can you tidy up the commit history a bit please? Zigflow uses rebase so these commits will be brought into |
|
I like the idea of opening a thread on slack to gather opinions on that. I'll send a message there so we can discuss with users. Regarding this branch's commit history, thank you for heads up! I'll do that. |
Introduce zigflow test and pkg/testrunner for validate-run-print flows on the zigflow-test queue, with per-machine file locking, failure output, docs, and e2e coverage. Test sessions use StartSession with skipScheduleUpdates and skipClientMetrics, terminate timed-out executions on the server, exclusive Windows file locks (overlapped lock file handle), and RunBytes for in-memory definitions. Signed-off-by: Arthur Costa <[email protected]>
3236a3b to
346a8c9
Compare
Summary
zigflow testto validate a workflow file, run it once against Temporal with JSON input, print the result, and exit with a non-zero code on failure (#304).pkg/testrunnerandcmd/run/session.gofor a short-lived in-process worker on the fixedzigflow-testtask queue (separate fromdocument.taskQueueused byzigflow run).raise) plus an error line and Temporal UI inspect link.examples/test-concurrency.Design decisions (discovered while implementing this feature, thus not present in #304)
These choices came out of implementation and e2e testing, not from the original issue text alone.
Fixed
zigflow-testtask queue (v1)zigflow testalways registers workers and starts workflows onzigflow-test, ignoringdocument.taskQueuein the YAML for this command. That keeps test runs off the same queue as long-livedzigflow runworkers, so a dev server withzigflow runup does not need to be stopped to try a workflow once. v1 prioritises predictable local/CI behaviour over honouring the file’s queue name; overriding viaregistrationTaskQueueinrunOptionsis the internal hook if we extend this later.Cross-process file lock on
zigflow-testMultiple
zigflow testprocesses can connect to the same Temporal server and namespace. Each process registers only its own workflow type on the sharedzigflow-testqueue. Temporal does not guarantee that a poll goes to a worker that registered that workflow, so concurrent tests can fail with “workflow type not registered” style errors.The lock is keyed by Temporal address + namespace (file under the user cache dir). It serialises who may poll
zigflow-testat a time for that pair. Different namespaces or servers do not block each other. This is process-level coordination, not an in-memory mutex inside one binary.Reuse
zigflow runworker startup viaStartSessionStartSessionbuildsrunOptionsand calls the sameprepareRegistrations,initTemporalClient, andstartInitialWorkerspath aszigflow run, without watch mode or signal handling. That avoids duplicating worker registration, codec, and external storage wiring. Test-specific behaviour is limited to flags onrunOptions(below).skipScheduleUpdatesfor test sessionsSchedule reconciliation (
UpdateSchedules) is appropriate for long-livedzigflow rundeployments. A one-shot test should not create or mutate Temporal schedules for the workflow under test. Test sessions setskipScheduleUpdates: true.skipClientMetricsand Prometheus (Temporal client metrics only)zigflow runenables Temporal SDK client metrics viatemporal.WithPrometheusMetrics(..., nil). Anilregistry uses Prometheus’s process-wide default registerer. Each call creates a new reporter that registers the same metric names again.That is fine for one
initTemporalClientper process (zigflow run). It breaks when several short-lived sessions run in the same process: central e2e (go testin one binary), multipleStartSessioncalls, or unit tests that invokeinitTemporalClientwithout skipping metrics. The second registration collides on the default registry (helpers/Tally behaviour; seezigflow/helpersPrometheus tests).Test sessions therefore set
skipClientMetrics: true. Workers in Zigflow do not currently setworker.Options.MetricsHandler, so this is not a client-vs-worker double registration inside one session; it is repeated client setup in one process.We did not adopt a process-wide shared
PrometheusHandleror per-sessionprom.NewRegistry()in this PR:zigflow testdoes not expose a stable metrics scrape target, and sharing would need explicit refcount/Close()lifecycle. Skipping client metrics for test is the smallest correct fix.initTemporalClientunit tests also setskipClientMetrics: trueso thecmd/runpackage’sgo testdoes not hit the same collision.zigflow runbehaviour is unchanged (metrics still enabled once per process).No “non-local Temporal” warning
Earlier feedback on #304: do not warn when the Temporal address does not look “local”. Cloud, in-cluster, and custom endpoints are all valid for
zigflow test; guessing locality is misleading.Failure output and e2e assertions
Failed runs print workflow output when the SDK returns it (useful for
raiseand OWS-shaped errors), plus the error and a Temporal UI history link. E2e assertions for the failure path were relaxed to matchApplicationErrorshape rather than assuming a specific string in the CLI error line alone.Test plan
go test ./pkg/testrunner/... ./cmd/...go test ./...task e2e(central e2e includingTestE2EZigflowTestandTestE2EZigflowTestFailed)temporal server start-dev,go run . test examples/hello-world/workflow.yaml --input examples/hello-world/test-input.jsongo run . test examples/raise/workflow.yamlexamples/test-concurrency/to verify lock serialisation