Skip to content

Uptrack-App/shutterpipe

Repository files navigation

shutterpipe

Uptrack probe-node sidecar — visual-regression diff + Cloudflare R2 artifact pipeline, in Zig.

What it does

shutterpipe runs alongside the Playwright Node sidecar (browser-sidecar) on Uptrack probe nodes (nbg1–nbg4) and handles two jobs that probe-rs (the Rust probe) delegates per browser check:

Subcommand Job
shutterpipe diff Perceptual diff between two screenshots — mean SSIM over 8×8 blocks, a 64-bit average perceptual hash, an 8×8 changed-region count, and an optional red-highlight diff overlay PNG. Drives visual-regression alerts.
shutterpipe upload Bundle a check run's per-step artifacts into one gzip'd tarball and PUT it to Cloudflare R2 (SigV4-signed, with retry). One PUT instead of N — the dominant R2 cost at scale.

Both subcommands read a JSON request from stdin and write a JSON response to stdout, so probe-rs can drive them via tokio::process::Command.

Implementation

Self-contained — no third-party Zig packages:

  • PNG decode/encode: vendored stb_image/stb_image_write single-headers (public domain), @cImported (same C-FFI pattern as the libcurl R2 client). Compiled from vendor/stb/stb_impl.c by build.zig.
  • Diff math (src/pixels.zig): pure Zig over RGBA buffers — SSIM, average-hash, region map, overlay, ignore-region masking. No I/O, unit-tested with synthetic images.
  • AWS SigV4 (src/sigv4.zig): pure std.crypto (HMAC-SHA256 + SHA256), validated against the AWS-documented canonical-request hash and an independent reference.
  • tar (src/tar.zig): minimal reproducible ustar writer; gzip (src/compress.zig) via std.compress.flate (std's zstd is decompress-only, so gzip avoids a libzstd C dependency).
  • R2 PUT (src/r2_curl.zig): HTTPS via system libcurl (per the openspec "TLS via libcurl, not std.crypto.tls" decision).

Build

zig build              # binary → zig-out/bin/shutterpipe
zig build test         # unit tests
zig build run -- diff  # run a subcommand locally

Targets Zig 0.16+ (uses std.process.Init, std.Io.File, the Writergate reader/writer; see ZIG016-DRIFT.md).

Subcommand contracts

diff

Request (stdin) — the baseline comes from a local file or an R2 key:

{
  "new_path":        "/tmp/run-456/step_03/screenshot.png",
  "baseline_path":   "/tmp/run-123/step_03/screenshot.png",
  "baseline_r2_key": "prod/mon_abc/baseline/screenshot.png",
  "r2_bucket":       "uptrack-browser-artifacts-prod",
  "ignore_regions":  [{ "x": 0, "y": 0, "w": 300, "h": 40 }],
  "threshold_ssim":  0.97
}

baseline_path (local) takes precedence; otherwise the baseline is fetched from R2 at baseline_r2_key (creds from env, see upload below). Auto-seed: if the R2 baseline is absent (404), the new screenshot is PUT as the baseline and the run passes ("baseline_seeded": true) — the first successful run establishes the baseline. A later run scoring below threshold_ssim fails the check, driving the visual-regression alert. Promoting an intentional change = overwrite the key (an API/FE action).

Response (stdout, exit 0; exit 1 on a read/decode/fetch error, 2 on a bad request):

{
  "ssim":            0.9923,
  "perceptual_hash": "e3a1b0c4…",
  "diff_count":      12,
  "passed":          true,
  "diff_image_path": "/tmp/run-456/step_03/screenshot.diff.png"
}

diff_image_path is null when the check passes or the dimensions differ. A seed run returns {"passed": true, "baseline_seeded": true, "diff_image_path": null}.

upload

Request (stdin):

{
  "monitor_id":   "mon_abc",
  "check_run_id": "run_xyz",
  "r2_bucket":    "uptrack-browser-artifacts-prod",
  "r2_prefix":    "mon_abc/run_xyz/",
  "expires_at":   "2026-06-23T00:00:00Z",
  "steps": [
    { "index": 0, "screenshot": "/tmp/.../shot.png", "console": "", "har": "", "dom": "" }
  ]
}

Response (stdout):

{
  "uploaded":    true,
  "tarball_key": "mon_abc/run_xyz/artifacts.tar.gz",
  "summary_key": "mon_abc/run_xyz/summary.json",
  "bytes":       214592,
  "duration_ms": 182
}

R2 credentials come from the probe node's environment (never the request):

Env Meaning
R2_ACCESS_KEY_ID R2 access key
R2_SECRET_ACCESS_KEY R2 secret key
R2_ENDPOINT <account>.r2.cloudflarestorage.com

With credentials unset, upload still bundles the artifacts and reports the keys/size with "uploaded": false, "reason": "r2_not_configured" (exit 1) — so the pipeline is exercisable locally; the live PUT is deploy-gated. Set SHUTTERPIPE_TAR_OUT=/path/out.tar.gz to also dump the assembled bundle for inspection (gunzip -c out.tar.gz | tar tf -).

Deploy

Installed on the probe nodes (nbg1–nbg4) via the NixOS module; probe-rs invokes the binary from PATH for browser checks that capture screenshots.

License

Apache-2.0 — see LICENSE.

Status

diff and upload are implemented and tested (27 unit tests + format cross-checks against system tar/gunzip and an independent SigV4 reference). The live R2 PUT is deploy-gated (needs node credentials). Remaining/optional: per-tier WebP recompression of screenshots, wiring probe-rs to call diff/upload per check, and baseline management. Tracked in add-synthetic-browser-checks.

About

Uptrack probe-node Zig sidecar: visual regression diff + Cloudflare R2 artifact pipeline

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors