Mob.Audio output probes: verify sound is actually working#54
Conversation
Mob can verify visual output in-process via screenshot/3, but had no
equivalent for audio. This adds two read-only probes that answer "is
sound actually coming out right now":
- Mob.Audio.output_status/0 — {volume, muted, route, other_audio}.
Cheap, no permission. Catches the common silence causes (muted,
volume 0, dead route). iOS AVAudioSession / Android AudioManager.
- Mob.Audio.output_level/1 — {rms_db, peak_db} | :silent | {:error, _}.
Reads actual signal energy, the part output_status and `adb dumpsys
audio` cannot answer. source: :mix (default) taps the global output
mix so it sees native players that bypass Mob.Audio (e.g. a game's
own AudioTrack); source: :mob taps Mob.Audio's own player.
Native wiring mirrors screenshot/3 + open_settings/1: -export/-nifs/stub
in mob_nif.erl, native table entries in mob_nif.zig and mob_nif.m, and
cacheOptional + null-guard for the app-owned Android bridge methods so a
drifted MobBridge.kt no-ops instead of crash-looping boot. iOS level-2
uses AVAudioPlayer metering (self-contained); Android uses a Visualizer
on session 0 (needs RECORD_AUDIO). output_level is a dirty IO NIF.
Honest asymmetry: for audio that bypasses Mob.Audio, level-2 works on
Android (global-mix tap) but not iOS (sandbox forbids it) — documented,
not papered over. Decoders are pure Elixir and unit-tested on host; the
Android Kotlin bridge methods ship in the mob_new template (separate PR).
Host: 969 mob tests pass, formatters clean. Device verification (boot +
live level reads) is the gate for the native paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
Companion Kotlin bridge methods: GenericJam/mob_new#25. Merge together. |
Device verification on a moto g power (2021), Android 11, corrected the
design: a session-0 (global output mix) Visualizer fails with ERROR_NO_INIT
for a normal app even with RECORD_AUDIO + MODIFY_AUDIO_SETTINGS — global
output capture is privileged. So the global-mix `:mix` path can't work in
the core framework.
Revised output_level/1:
- default source is now :mob — meters Mob.Audio's own player. iOS via
AVAudioPlayer metering; Android via a Visualizer on the player's OWN
audio session (audioPlayer.audioSessionId), which works with RECORD_AUDIO.
- :mix returns {:error, :unsupported_on_platform} on both platforms. True
global/foreign-app capture is deferred to a separate MediaProjection-based
plugin (test-env dep) — see the decision doc.
The Android NIF now decodes a length-coded return (float[2] = level;
float[1] = error code 1 unsupported / 2 needs_record_audio / 3 not_playing).
Verified on hardware (dist-RPC into doom_demo):
output_status → %{route: :speaker, volume: 0.2, muted: false}
output_level(:mob) → {-34.8, -31.8} playing; {:error, :not_playing} idle
output_level(:mix) → {:error, :unsupported_on_platform}
no RECORD_AUDIO → {:error, :needs_record_audio}
app boots cleanly (NIF table OK)
output_status/0 is unchanged and fully verified. Companion template update:
mob_new (own-session Visualizer, no session-0 tap).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Review — approve with follow-ups (FFI seam clean, drift-safe)Traced both new NIFs ( Two semantic gaps worth tracking (both verified in the diff, not blockers): 1. 2. Minor: the title "verify sound is actually working" is a bit stronger than what two probes measure (signal-present + config-sane, not emission-at-the-speaker) — the module docs are honest about this, so just don't let the title over-set expectations. Merge with mob_new #25 (Android needs its bridge methods). CI green, base current. (FFI parity, |
Integrates mob#67 (input-level metering, now on master) with the output probes on this branch: - audio.ex: drop the duplicate decode_level/1 the merge produced (the shared helper now lives once; output_level + input_level both use it). - audio_test.exs: keep decode_status/1 tests; unify decode_level/1 to cover atoms from both probes (:not_playing/:needs_record_audio, :not_metering, :unsupported_on_platform). - ios/mob_nif.m: forward-declare g_audio_player so output_level/1 compiles (it referenced the play/1 player ~620 lines before its definition — a latent iOS build break independent of the merge). Elixir compiles --warnings-as-errors; 30 audio tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
A build-time `deps` symlink (→ an absolute local path) was accidentally staged by `git add -A` in the integration worktree and rode the #54 squash into master. It's a self-referential absolute-path symlink that breaks any other checkout. Untrack it; deps/ is regenerated by `mix deps.get` and is gitignored (/deps/). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- Mob.Audio.output_status/0 + output_level/1 — verify sound is actually playing (#54) - Mob.Audio.start_input_metering/1 + input_level/0 + stop_input_metering/1 — agent ears (#67, MOB-35) Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
What
The audio analog of
screenshot/3: read-only probes that answer "is sound actually coming out right now."Mob.Audio.output_status/0→%{volume, muted, route, other_audio}. Cheap, no permission. Catches the common silence causes (muted, volume 0, dead route). iOSAVAudioSession/ AndroidAudioManager.Mob.Audio.output_level/1→{rms_db, peak_db}|:silent|{:error, reason}. Reads actual signal energy — the partoutput_statusandadbcan't answer.source: :mob(default) meters Mob.Audio's own player (iOSAVAudioPlayermetering; AndroidVisualizeron the player's own session, needs runtimeRECORD_AUDIO).source: :mixreturns{:error, :unsupported_on_platform}.Scope correction from device verification
The original design had
:mixtap the global output mix to catch audio from native players that bypassMob.Audio(a game's ownAudioTrack). Device testing on a moto g power (2021), Android 11, disproved this: a session-0Visualizerfails withERROR_NO_INITeven withRECORD_AUDIO+MODIFY_AUDIO_SETTINGS— global output capture is privileged (signatureCAPTURE_AUDIO_OUTPUT, or MediaProjectionAudioPlaybackCapture). iOS forbids it by sandbox.So in-app these probes verify your own audio. True global/foreign-app capture is deferred to a separate MediaProjection-based capture plugin intended as a test-environment dependency (Android-only). For "is a bundled game's audio working" without that plugin,
adb shell dumpsys media.audio_flinger(active track + underruns) is the answer.Device verification (moto g power 2021, Android 11) ✅
dist-RPC into a real build:
output_status/0→%{route: :speaker, volume: 0.2, muted: false, other_audio: false}.output_level(:mob)while a looped tone played →{-34.8, -31.8}; idle / after stop →{:error, :not_playing}.output_level(:mix)→{:error, :unsupported_on_platform}.RECORD_AUDIOgranted →{:error, :needs_record_audio}.How
Native wiring mirrors
screenshot/3+open_settings/1:-export/-nifs/stub inmob_nif.erl, native table entries inmob_nif.zig+mob_nif.m,cacheOptional+ null-guard for the app-owned Android bridge methods (drift-safe boot).output_levelis a dirty IO NIF. Android returns a length-codedfloat[](len 2 = level; len 1 = error code) the NIF maps to atoms; decoding (route codes,:silent/:error) is pure Elixir (decode_status/1,decode_level/1), unit-tested on host. Companion Android Kotlin: GenericJam/mob_new#25.Host: mob test suite green, formatters clean (mix format, erlfmt, clang-format, zig fmt). Decision record + follow-up plugin scope:
decisions/2026-06-29-audio-output-probes.md.🤖 Generated with Claude Code