v1.9.0
kevinraymond/fosforav1.9.0Jul 17, 2026by github-actions[bot]
AI Summary
A major audio engine overhaul featuring auto-reconnect, structure detection, new visual effects, and shader ABI updates.
Key Highlights
- Audio auto-reconnect with exponential backoff
- Musical key detection with constant-Q chromagram
- Build-up and drop detection for structure analysis
- New effects: Beam (CRT oscilloscope) and Strata (spectral canyon)
- Shader ABI v2 expansion for new audio features
New Features
- Audio auto-reconnect and default output device follow
- Smooth beat and bar phase interpolation
- Tempo prior control with genre presets
- EBU R128 loudness detection
- Downbeat and meter tracking
- Audio textures (waveform, spectrum, spectrogram)
- Beam and Strata visual effects
- Mel bands as binding sources
- OSC Broadcast-TX toggle
Full Release Notes
### Added - **Audio auto-reconnect** — the stall watchdog was detection-only: it raised a toast and left you to re-pick the device by hand. A confirmed device death now reopens the capture automatically, backing off 1/2/4/8 s across five attempts, with a new **AUD** status-bar dot reporting live, quiet, reconnecting or failed. Teardown runs off the render thread, so a backend stuck in a blocking read can't freeze the visuals. On by default; a destroyed PipeWire capture node is back with live data in under a second. - **Follow the default output device when it changes (Linux)** — the common case on Linux isn't a device death, it's switching your system output mid-set, and PipeWire silently *migrates* the capture stream to the new default sink's monitor with no error and no stall — so the watchdog is structurally blind to it and the app goes on capturing the wrong device while looking perfectly healthy. The default sink is now polled every 5 s and the capture reopens onto whatever is current, within ~4 s end to end. Reuses the auto-reconnect setting; the Windows equivalent is a follow-up. - **Smooth beat and bar phase at render rate** — the render thread took the newest 86.1 Hz analysis frame as-is, so on a 120–144 Hz display ~28% of frames reused an identical feature vector and phase-locked motion visibly stair-stepped, with a dropped frame reading as a phase pop. Features are now interpolated between frames, and both `beat_phase` and `bar_phase` run on a phase-locked loop advanced every render frame. Duplicate `bar_phase` frames went 10.4% → 0%, with tempo holding to 1.998 wraps/sec against 2.000 expected. Pulse timing is unaffected. - **Tempo prior control — genre presets, auto-adapt, tap tempo, half/double** — the prior that decides which octave a track reads at was hardcoded at 150 BPM, so a 172 BPM drum-and-bass track could fold to ~86 with no way to say otherwise. Adds six genre presets (Neutral, Wide, House, Drum & Bass, Hip-hop, Ambient), an auto mode that walks the centre toward the tempo actually being locked, ×2 / ÷2 overrides and tap tempo — all MIDI/OSC-mappable and applied live. The default reproduces the exact previous behavior, so upgrading changes nothing until you pick a preset. - **Musical key detection + true constant-Q chroma** — the chroma wheel hard-rounded every FFT bin to the nearest of 12 pitch classes (its "constant-Q" tooltip was simply false). It is now a real constant-Q chromagram with a slow tuning estimator, so a 432 Hz-tuned track no longer smears across pitch classes. A rolling correlation against the 24 Krumhansl-Kessler profiles reports the key with hysteresis, so it doesn't flicker between relatives. The audio panel gains a key readout (`C maj · 82%`) and the key is broadcast over OSC. - **Build-up / drop / section-boundary detection** — the engine can finally see structure beyond a single beat, so the drop no longer has to be hand-triggered. `buildup` is a 0..1 tension ramp blending loudness rise, spectral brightening, onset density and the classic pre-drop sub-bass withdrawal; `drop` is a one-frame pulse fired when a sustained build-up breaks into a loudness jump with the sub-bass returning, with a 16 s refractory; `section_novelty` peaks where the arrangement's block structure changes. Heuristics tuned for electronic music. - **Build-up / drop tuning panel** — those thresholds were hardcoded with no user access. Ten knobs are now live-tunable from a **TUNING · A18 build/drop** section in the audio panel, applied with no pipeline rebuild, no capture gap and no counter reset, persisted to `settings.json` and surviving a device switch. - **EBU R128 / BS.1770 loudness — `loudness_m` / `loudness_s` / `loudness_trend`** — proper K-weighted momentary (400 ms) and short-term (3 s) LUFS, with filter coefficients re-derived for the actual device sample rate. `loudness_trend` is the rising component, a ready-made build hint. Also establishes a perceptual **silence gate** (momentary < −55 LUFS) that the other detectors now share, so silence is judged the same way across devices and content. - **Downbeat / bar-phase / meter tracking** — a tracker sitting after beat detection scores 3/4 against 4/4 and each candidate bar phase by how much more "downbeat-like" the beats at that phase are, locking the winner with ~8-beat hysteresis and falling back to 4/4 aligned to the strongest recent beat when confidence is low. `downbeat` fires on the bar's one, `bar_phase` is a 0→1 sawtooth, `beat_in_bar` the normalized index. Roughly 70–80% accurate on 4/4 electronic music. - **Audio textures — waveform / spectrum / scrolling mel-spectrogram** — three shader textures that shipped as 1×1 placeholders now carry real data: `waveform(x)` is a zero-crossing-triggered min/max PCM window (so the trace holds still), `spectrum(x)` a log-frequency magnitude spectrum, and `spectrogram(uv)` a scrolling 64-band mel history. Unlocks oscilloscopes, spectrum bars and waterfalls. - **Beam — vector-CRT oscilloscope effect** — the first effect to draw the audio signal itself. Gaussian beam energy is integrated along the waveform polyline and dwell-weighted by inverse screen length for the signature bright-slow, dim-fast CRT look, rendered into a slow-decay feedback pass that acts as the phosphor persistence. Two modes: scope (horizontal sweep with a min/max envelope) and radial (waveform wrapped on a circle whose radius grows with `rms`). `centroid` sets colour temperature, `zcr` beam focus, `beat` persistence kick. - **Strata — spectral-canyon effect** — a heightfield flown over the last ~8 s of mel history: the audio *is* the terrain, loud making ridges and quiet chasms, with lateral position as frequency and the newest audio erupting nearest the camera before ageing into the distance. Soft raymarched shadows, sun-tinted height fog, and height and slope materials with snow caps that follow loudness. Six params including a zoom for the overview. `rolloff` drives draw distance, `flatness` gloss, `beat` and `kick` ridge glow. - **Mel bands as binding sources** — the 64-band mel column is now exposed as `audio.mel.0..63` binding sources, so any effect parameter can be driven by an individual band. Binding-only, not broadcast over OSC (64 args per frame would be heavy). - **Audio panel STRUCTURE readout and an OSC Broadcast-TX toggle** — the new detectors are visible in-app without an external OSC monitor: short-term loudness, loudness trend, build-up, section novelty and a drop indicator that flashes on the pulse. Also fixes an OSC-panel gap — TX broadcast had host, port and rate fields but no enable checkbox, so it could only be turned on by hand-editing `osc.json`. - **Shader ABI v2** — one batched bump reserving every slot the detectors above needed, so each could land without further churn: `AudioFeatures` 46 → 61 features (184 → 244 B), `ShaderUniforms` 288 → 352 B, plus the audio-texture bindings. Existing offsets are untouched, but compiled user shaders need recompiling once for this bump. Effects can bind the reserved fields immediately and light up automatically as each detector ships. - **Particle shaders gain five more audio features** — `flatness`, `rolloff`, `bandwidth`, `bpm` and `beat_strength`, filling reserved padding with no layout change, so compute shaders can react to spectral shape and tempo strength the way fragment shaders already could. - **Community health files** — `CODE_OF_CONDUCT.md`, `SECURITY.md` and a pull-request template. ### Changed - **Spectral feature correctness pass — centroid / flux / flatness / rolloff** — all four were subtly wrong. Centroid lived in the top octave and was unstable near silence; it is now a power-weighted mean of log-frequency across a musical 40 Hz–18 kHz range, usable as a brightness fader. Flux doubled when the volume doubled (it was effectively a second RMS) and is now level-invariant, measuring change rather than level. Flatness now cleanly separates tonal pads from noise sweeps. All six spectral-shape features are also broadcast over OSC — previously only `kick` was. - **Kick detection no longer double-normalized** — `kick` was auto-levelled twice, so it was nearly always saturated on quiet material and its scale depended on two interacting auto-levelers. It is computed once now, as a level-invariant log-magnitude flux against its own long-term percentile, gated on the perceptual silence flag. Kick-bound strobes stop firing on hi-hat bleed, bassless leads and quiet passages, and stop saturating on loud sustained bass. - **Gated percentile normalization with per-feature policies** — one symmetric running min/max was applied to every feature, so quiet-room noise stretched to full scale and visuals danced to silence, and a single transient spike squashed everything for ~2 s. Each feature now picks a policy: energy features use gated percentile ranging, frozen under perceptual silence so a quiet room can't be ranged up; known-range spectral features hold their last value through silence instead of dancing; MFCCs standardize symmetrically about 0.5. - **Unified band scaling** — the seven frequency bands were computed in two incompatible families, half linear RMS and half dB, so a visual bound to `band.0` and one bound to `band.5` behaved like different species and the normalizer saw wildly different input dynamics. All seven now share one dB domain with an equal-loudness tilt above 2 kHz. A **Settings → Band Scale** dropdown keeps `Legacy` available, reproducing the exact previous behavior for presets tuned to the old feel. - **SuperFlux onset detection closes the 250–500 Hz snare gap** — the onset stage used four fixed bands with a hole at 250–500 Hz (snare bodies, toms, male vocals), and false-triggered on vibrato and pitch slides. It is now SuperFlux: a 64-band contiguous filterbank whose reference frame passes through a frequency maximum filter, so a partial can drift slightly between frames without registering an onset. The kick/snare/hat balance is preserved and tempo tracking downstream is undisturbed. - **Deterministic 512-sample analysis hop** — the audio thread slept 10 ms and analyzed whatever had arrived, so the effective hop varied with scheduler jitter and grew under load: flux amplitudes moved with hop size and the tempo estimator had to guess its own frame rate from wall-clock timestamps. It now runs exactly one frame per 512 samples on a sample clock, so timing is exact even when a burst of hops is processed in one wakeup, and the estimator's frame-rate guessing is gone. - **Audio feature schema as a single source of truth** — normalization, smoothing and stale-decay policies lived in a hand-ordered 46-entry table and positional index literals, so adding a feature risked silently shifting indices out from under those stages. All three now read one ordered table, pinned to the struct layout by a compile-time assertion. No behavior change. ### Fixed - **`beat_phase` dropped to 0 several times a beat on loud audio** — the phase-freeze gate tested normalized `rms`, which floors at exactly 0.0 whenever the signal touches the bottom of its own recent range — on rhythmic material, the trough between every hit. So it fired on perfectly loud audio, manufacturing false wraps: a 120 BPM signal read 2.12 wraps/sec instead of 2.00. It now gates on the perceptual silence flag, which a loud trough can't trip. Spurious zeros 164 → 0. - **`bar_phase` swept through silence while `beat_phase` sat frozen** — the beat detector took the perceptual silence gate and pinned its phase at 0; the downbeat tracker never took it, so through a quiet passage the bar clock kept ramping against a dead beat, and any bar-synced visual inherited the mismatch. Verified live on a silent monitor: across 361 samples `beat_phase` held a single value while `bar_phase` swept the full 0→1. - **Quitting during an audio stall could hang the process** — shutting down dropped the capture inline, which joins a thread that may be blocked in a timeout-less read, so the join never returned and the app hung on exit rather than closing. It now hands the backend to the detached reaper and exits. - **PulseAudio read errors retried forever at 10 Hz** — once the server kills a `pa_simple` stream every subsequent read fails, and the capture loop had no exit path, so a dead stream pinned a thread logging an error ten times a second for the rest of the session. It now gives up after ~1 s and publishes a failure the watchdog can act on. A successful read resets the counter, so transient errors are unaffected. - **A device switch stranded an in-progress recording's audio** — switching installed a fresh recording ring, but a recording started earlier holds the old one, so its writer was left draining a ring nobody writes to and the rest of the take recorded silence. The ring is now threaded through the reopen. Note that a recording still captures the sample rate at start, so reopening onto a device with a different rate pitch-shifts the remainder. - **A stale or foreign pipeline cache no longer aborts startup** — a `pipeline_cache.bin` written by a different GPU or driver (a GPU swap, hybrid graphics, an eGPU, or a driver update) made the first pipeline creation fail, so the app died with "Failed to initialize app" and a black launch. The cache is now validated on load, and discarded and rebuilt if the driver rejects it. - **BPM test harness fed frames at the wrong clock** — the synthetic convergence tests spaced kicks at 100 Hz while the detector had moved to an 86.1 Hz hop, so every target tempo reached the estimator 13.9% low (a nominal 172 arrived as 148). The ±15% tolerance bands were wide enough to hide it, so the tests passed while asserting something other than what they claimed. ### Documentation - **README and TUTORIALS rewritten for newcomers** — a plain "What is this?", a 3-step quick start with first-10-seconds expectations, a "Make it yours" section, and an FAQ covering audio-input selection, black screens and macOS notarization. Also corrected the effects lists (the README listed 8 of the old set; TUTORIALS listed four effects that don't exist), the keyboard table, the build prerequisite and the audio band ranges. ## Downloads | Platform | File | Notes | |----------|------|-------| | **macOS** | `.dmg` | Universal binary (Apple Silicon + Intel) | | **Linux x86_64** | `.tar.gz` | glibc 2.35+ (Ubuntu 22.04+, Debian 12+) | | **Windows x64** | `.zip` | Windows 10+ | ## Getting started **macOS**: Open the DMG, drag Fosfora.app to Applications (or run directly). Signed and notarized. **Linux / Windows**: Extract, run `fosfora` from the extracted directory. The `assets/` folder must be next to the binary. ## Requirements - **GPU**: Vulkan (Linux/Windows) or Metal (macOS) - **Audio**: Built-in mic or line-in for audio-reactive visuals - **NDI®** (optional): [NDI® SDK runtime](https://ndi.video) for network video output - **Video playback** (optional): `ffmpeg` on PATH for media layers --- NDI® is a registered trademark of Vizrt NDI AB.