dotbeat

A git-native, agent-native DAW. Your song is a diff-friendly text file you — and an AI agent — can edit in a real GUI, render from the command line, git diff, and get sound-design critique on.

Solo project, built in paired sessions with Claude. The .beat text format is the product: a real GUI (dotbeat's own, not a wrapped teaching tool), a CLI (beat), and an MCP server for AI agents all edit the same file, and every edit is a one- or two-line git diff — not a binary blob you hope merges.

npm install
npm test                          # 1,426 tests: format, conversion, daemon sync, CLI surface, vary/humanize, DSP metrics, MCP parity
node cli/beat.mjs init song.beat --bpm 124 && node cli/beat.mjs add-track song.beat drums drums
node cli/beat.mjs inspect examples/real-groove.beat
node cli/beat.mjs set examples/real-groove.beat bass.cutoff 900   # prints "bass: cutoff 700 -> 900"
node cli/beat.mjs quantize examples/real-groove.beat bass --grid 1 --amount 0.6   # partial-strength snap to 16ths
node cli/beat.mjs vary examples/real-groove.beat drums kick --render             # small-diff variants to audition...
node cli/beat.mjs score vary-kick-42 3 7 1                        # ...ranked picks -> git-tracked taste log
node cli/beat.mjs diff --git HEAD~1 HEAD song.beat                # a musical edit list, not line noise
node cli/beat.mjs render examples/real-groove.beat -o out.wav     # dotbeat's own engine, headless
node cli/beat.mjs metrics out.wav                                 # LUFS (BS.1770), true peak, crest, spectrum, stereo
node cli/beat.mjs checkpoint song.beat --label "rough mix"        # a restorable version, git-backed
node cli/beat.mjs macro apply song.beat bass space 50              # a knob that resolves to real literal edits, no indirection
node cli/beat.mjs mcp                                             # the whole toolchain as MCP tools for an AI agent
node cli/beat.mjs daemon examples/real-groove.beat                # two-way sync backend for the GUI

cd ui && npm install && npm run dev   # dotbeat's own frontend (Vite + React), talks to the daemon above

Everything above runs with zero Python. The ML-backed commands shell out to a fleet of eight Python sidecars (beat/downbeat detection, text-to-audio generation, Surge XT patch rendering, roughness curves, audio embeddings, MIDI figure extraction, Demucs stem extraction, Composer's Assistant 2 — the full table is in python/README.md); all of them share one spawn/doctor contract and degrade to actionable errors, and CI runs green with zero Python via stub backends. The two you need first — beat analyze (real beat/downbeat detection) and beat source gen (text-to-audio one-shots: Stable Audio Open locally, or hosted on fal.ai with --backend fal, where a provider adapter table also covers Stable Audio 2.5/3, Lyria 2, MiniMax Music, and ElevenLabs Music, plus BPM-aware downbeat trimming and per-role stem extraction) — need a Python 3.10 venv at python/.venv and, for local generation, a Hugging Face login + accepting the gated model license at stabilityai/stable-audio-open-1.0:

brew install [email protected] && /opt/homebrew/bin/python3.10 -m venv python/.venv
python/.venv/bin/pip install -r python/requirements-beatthis.txt -r python/requirements-stableaudio.txt
python/.venv/bin/hf auth login                     # token from huggingface.co/settings/tokens
node cli/beat.mjs analyze --doctor && node cli/beat.mjs source gen --doctor   # both should report ok

Step-by-step details (why 3.10, what each pinned dep fixes): python/README.md. For what this unlocks, see examples/recipe-song/ — a complete track in which every audio asset was generated by beat source gen: one bell one-shot pitch-mapped into a melody across six declared drum lanes, a found-sound kit whose snare was picked from three seed candidates by measured spectral centroid, and a pad repitched into key with a one-token audio-clip edit. Each media/*.wav carries a provenance sidecar recording prompt/seed/model, and regenerating from those fields reproduces the registered file byte-for-byte (sha256-verified) — the .beat file is literally a recipe.

Run node cli/beat.mjs with no arguments for the full command list — it's grown a lot (track groups, drum kits, audio-clip splitting, effect chains, checkpoint/restore/pin history, selection sync with the GUI, production tricks, per-section mix feedback, blind source showdowns, the beat board review surface) since the walkthrough above.

   song.beat  (plain text, in git)
      │
      ├─►  GUI            edit notes/knobs/arrangement → one-line diffs
      ├─►  CLI            beat render / inspect / set / diff / vary / checkpoint
      ├─►  AI agent        edits the file, renders, critiques, proposes diffs (MCP)
      └─►  git             branch, diff, merge, review — like code

Nobody currently holds all of GUI + diff-friendly-text-file + CLI/agent at once — openDAW has the GUI and a headless engine but zips its projects into an opaque binary bundle; REAPER has genuinely text project files but even its own practitioners say git "cannot meaningfully diff or merge" them; Ableton's .als is gzipped XML with no native diff story at all; Strudel/Tidal are text-native but have no GUI. This project aims at that gap. See ROADMAP.md for the full thesis and prior-art comparison.

The core loop this project was built to prove works and is exercised daily: a hand-inspectable .beat file is the source of truth for a live GUI session, a CLI, and an AI agent, all at once — turn a knob in the GUI and git diff shows exactly one changed line; edit the file by hand and the GUI hot-reloads without stopping playback.

Built and stable (177 of 375 tracked features done, 9 more in progress — the live list is docs/product-roadmap.md):

• The format and edit surface — round-trip parser/serializer, semantic diff, edit primitives (quantize/humanize/transpose/groove/fit-to-scale), track groups, drum kits, audio clips with splitting, curved automation, and macros that resolve to real literal edits.
• dotbeat's own GUI — arrangement view, piano-roll/drum-lane clip editing, mixer, device panel, content browser, in-session undo/redo alongside the git-backed checkpoint/restore history. Its design was shaped by three research passes against Ableton Live 12's own reference manual and hardened by ~30 exploratory usability pilots.
• CLI/MCP parity — 71 MCP tools mirror the project-editing toolchain for AI agents, with per-command --help. Parity is increasingly structural rather than reviewed-in: a tools/list golden snapshot, a table-driven CLI↔MCP byte-parity test, a CLI surface test (help↔dispatch integrity, unknown-flag rejection), and shared orchestrators like runVaryBatch. The daemon, the human review surfaces (beat rate, beat board), and the owner-side taste/eval loop (showdown, prodtask, pilot, taste-*) stay deliberately CLI-only.
• Render and guardrails — headless renders through dotbeat's one canonical engine (live-capture by default; --offline computes the same mix through an offline context, and batch renders default to offline where it's both exact and fast), BS.1770 LUFS / true-peak / spectral / stereo metrics, and beat lint mix critique against reference profiles derived from music the owner loves — now joined by per-section energy-arc feedback (beat feedback --sections, arc reference profiles), a calibrated pathology-screen suite (clicks, silence, arrangement flatness), and a psychoacoustic roughness ear. Clip automation renders truthfully — automation lanes win over static patch values (the research/121 stomp bug, fixed and golden-WAV-gated).
• The agent↔owner loop — beat board (a non-blind picking UI over undecided batches, with a separate decisions log that never mixes with the blind scores log), beat open (deep-link the running GUI to a track/param), beat diff --since --rollup (the agent's morning read of what the owner changed), and opt-in edit telemetry across all three write surfaces (research/128, D29).
• ML sidecars, contained — eight Python sidecars behind one shared spawn/doctor scaffold (src/analysis/spawn-sidecar.ts): beat analyze (Beat This beat/downbeat detection), beat source gen (Stable Audio Open locally; hosted providers via a fal.ai adapter table — Stable Audio 2.5/3, Lyria 2, MiniMax, ElevenLabs — with downbeat trimming and Demucs stem extraction), Surge XT patch rendering, roughness curves, Audiobox-Aesthetics embeddings (CLAP measured below chance and retired from the default), MIDI figure extraction, and Composer's Assistant 2. Python stays tiny and dumb; TypeScript owns caching/hashing/contracts; stub backends keep CI green with zero Python; owner-side tests cover the real-model paths CI structurally can't. The proof: examples/recipe-song/ regenerates every audio asset byte-for-byte from provenance sidecars.
• The generative sampler — beat sample-info pitch detection, beat keymap minting a playable, diffable sampler instrument from any one-shot, and generation candidates flowing through the same score/adopt loop as everything else.

The active frontier: the taste-model program (docs/taste-loop-design.md) — teach a model the owner's musical taste from blind-rated batches, then let it steer search. The model is advisory by construction; the owner's picks stay the only ground truth. As of 2026-07-21, on 66 blind-rated batches (re-evaluated after fixing the CLAP fusion bug below): the DSP Bradley-Terry ranker predicts held-out picks at 33% top-1 / 62% pairwise (chance 21%/50%) — replicates the earlier 37-batch result; the four Audiobox-Aesthetics axes alone reach a similar 33% top-1 but a much stronger 81% pairwise on source-showdown batches specifically; DSP+Aesthetics combined is now the best scorer (36% top-1, 65% pairwise). CLAP embeddings, once correctly pooled, turned out to hurt rather than help — embed-bt scores below chance held-out, and adding CLAP to DSP makes the combined scorer worse than DSP alone (see docs/taste-loop-design.md "CLAP retired from the active ensemble" for the diagnosis). This doesn't touch the live beat suggest --taste tool, which was DSP-only already — it's an ablation-study-only finding. Batch renders are loudness-normalized by default so "louder" can't masquerade as "better." The data pipeline (taste-seeds → taste-collect → a blind web rating UI) was hardened by ear — six classes of silent no-op batches found and closed.

The generator question, decomposed by blind data (2026-07-21/22 eval arc — decisions D24-D27). beat showdown (blind per-role source comparisons, now 60+ rated batches) split the engine's deficit into two measured axes: production (width/air/motion — a production-only ablation moved the engine from 3% to 29% pairwise; the format had the DSP all along, nothing set it) and timbre (the T6 ceiling study: envelope solved, spectrum unreachable). The response, all built and blind-verified: produced defaults (src/analysis/produce.ts — gen-kit and taste-seeds ship with role-appropriate production), the beat trick catalog (15 named production moves with preconditions and measurable receipts — verified by ear via beat prodtask transform: tricked beat a random-edit control 90% vs 33% pairwise), Surge XT as an out-of-process sound factory (--with-surge; 639 factory patches; a surgepy channel-corruption bug was root-caused and fixed — upstream issue drafted), and commercial-MIDI figures (--midi-dir, private) that hold composition at commercial quality. Composition since grew two more figure sources that don't need private data: a theory composition layer (voice-leading chord generation + a motif-first lead generator with pre-render gross-error gates, --theory) and Composer's Assistant 2 (--ca2, composing over the chord track — the winner of the research/125 MIDI-model trials). Current standings with commercial figures: refs ~93% pairwise, gen 52% ≈ surge 50% ≈ keymap 45%, raw engine 0% — the chosen path (D26) is making engine/surge sound commercial, and the program's north star (D27) is the first blind batch where the owner genuinely ranks a dotbeat-rendered clip above the reference. T4 (beat suggest --taste) is live; the critic carries bootstrap-ensemble uncertainty (pessimistic scoring that auto-discounts its measured blind spots), and the first constrained T5 overnight pilot is built as beat pilot — critic-guided quality-diversity search over per-seed lineages, with beat board as the owner's morning review surface.

How it's kept honest. 1,426 tests plus a manifest-run Playwright verify fleet (npm run verify, with a verify:engine tier) assert known-correct behavior; exploratory usability pilots — an agent driving the real app or CLI with no checklist (docs/usability-testing.md) — keep finding what scripted suites structurally cannot, and their findings land as roadmap rows, never a second backlog. A six-stream codebase review (docs/research/130, 2026-07-25) turned "parity by review discipline" into structure: safety gates first (CLI surface test, MCP parity snapshot, committed engine golden WAVs, === guards on frozen eval constants), then the top-ranked extractions (one sidecar spawn scaffold, one seeded RNG, one figure vocabulary, one vary-batch orchestrator) and thousands of lines of dead code deleted. The phase-by-phase history that used to live in this section is in docs/ (phase plans, 130 research passes) and the git log.