Strudel → Hydra AV feature mapping: wiring the 4-bin FFT bridge

This module builds the hands-on fluency that precedes any coherent AV performance in the Strudel+Hydra rig: the ability to wire a band to a parameter, tune the response, and trust that what you hear is actually driving what you see. In a live set this fluency is pre-performance prep — you don’t debug thunks on stage — so the capstone is a complete, runnable reactive sketch built and stress-tested before the crowd arrives.

The arc starts with the bridge itself. The AV link is a single, narrow channel: Strudel’s Web-Audio analyser fills a 4-element array a.fft[0..3] on a shim that replaces Hydra’s native audio object. No MIDI, no OSC, no per-instrument bus. The first exercise confirms the channel by adding .analyze('hydra') to one Strudel voice, checking that a.fft[0] moves, then removing the tag and watching the visuals go dead — the tag requirement is easy to forget and responsible for the most common “why isn’t it reacting?” failure.

With signal flowing, the next layer is the thunk misconception: a.fft[0] * 2.5 is frozen at eval time; only () => a.fft[0] * 2.5 updates every frame. This is introduced early because it blocks everything downstream — a frozen parameter looks like a working sketch and is almost always the first bug a new learner hits. Pair it immediately with the out-of-bounds guard: a.fft[4] returns undefined, silently turning every arithmetic result into NaN and freezing or blacking out the frame.

From there the learner works through the shim’s four tuning knobs. setScale is the gain before the sketch; setCutoff subtracts noise floor; setSmooth sets one-pole smoothing globally; setBins re-slices bands (stay at 4). The canonical failure modes are the sanity-check atoms: if a parameter spends most of its time pinned at min or max, scale/gain/base are miscalibrated. The calibration drill — raise scale until the visuals breathe, lower smooth until motion is legible but not jittery — is the part-task drill that runs on every new patch.

Transfer functions are the creative core. Affine linear base + gain · a.fft[i] suits parameters that should always be moving (rotation, scroll). Squared-exponential base + gain · a.fft[i]² suits parameters that should stay near their floor when quiet and pop on loud hits — a proxy for onset detection that the display-rate FFT (~60 Hz, transients averaged away) cannot provide directly. The distinction between these two forms, and the understanding that the base term must produce an intentional-looking frame at silence, is tested explicitly in the capstone’s design rationale.

The mapping bank exercises build on the seed: bass → large/slow elements, highs → fine/fast detail (kaleidoscope symmetry being the documented example). The one-band-one-target discipline and the element-size-to-register match are the two rules that separate a mud patch (everything pumping together) from a legible spectral-band-split. The four-band-layout insight ties everything together: a well-budgeted Strudel mix organizes the spectrum so that bass carries the kick and highs carry the hats, which means the mapping assignments are not arbitrary — they key off the same spectral convention that makes the mix itself legible.

The amplitude-proxy and amplitude-pump-proxy atoms give the learner the two supported workarounds for onset-ish coupling: squaring a band value for soft punch, and an exp-curve bass envelope for sidechain-style ducking. These sit at the edge of what the bridge can do and are gated here rather than later because they require the transfer-function and sanity-check fluency from earlier in the arc.

The supporting atom punctual-av-bridge-collapse is enrichment: Punctual removes the bridge entirely because audio and visuals share one program. It is not a required stop on this arc but provides the correct conceptual contrast — the 4-bin shim is a design constraint specific to the Strudel+Hydra combination, not a universal fact about live-coded AV.

Walkthrough

The Strudel→Hydra bridge is narrow: a 4-bin FFT (a.fft[0..3] = bass, low-mid, high-mid, highs) is all the audio Hydra sees. You’ll start from the rig’s seed sketch and build it into a robust, expressive bridge — new couplings, shaped transfer curves, and the safety rails that keep it from pinning or crashing mid-set. Open the Hydra editor, run with Ctrl-Shift-Enter, and remember: the value in each () => … thunk is re-read every frame ([[hydra-reactive-thunk]]).

1 — the seed. The rig ships three couplings: bass (fft[0]) → brightness, high-mid (fft[2]) → rotation, low-mid (fft[1]) → warp. setBins(4) fixes the band count so the indices are stable. This is your known-good starting point ([[seed-mapping-formalized]], [[one-band-one-target]]).

a.setBins(4)
osc(20, 0.1, 0.6)
  .brightness(() => a.fft[0] * 0.4)          // bass → brightness
  .rotate(() => a.fft[2] * 3)                // high-mid → rotation
  .modulate(noise(2), () => a.fft[1] * 0.3)  // low-mid → warp
  .out(o0)

2 — a fourth coupling: highs → symmetry. The highs band (fft[3]) is unused. Wire it to kaleidoscope symmetry so hats and cymbals fracture the frame — matching a fast, bright band to a fast, structural parameter ([[highs-to-kaleid]], [[band-element-size-match]]).

a.setBins(4)
osc(20, 0.1, 0.6)
  .brightness(() => a.fft[0] * 0.4)
  .rotate(() => a.fft[2] * 3)
  .modulate(noise(2), () => a.fft[1] * 0.3)
  .kaleid(() => 2 + a.fft[3] * 6)            // highs → kaleidoscope symmetry
  .out(o0)

3 — shape the response (transfer functions). A linear map reacts to everything equally, so quiet detail muddies the visual. Squaring the signal (x²) suppresses the floor and pops only on peaks — an onset-ish response. The transfer curve is where a mapping gets its feel ([[av-transfer-function-forms]], [[amplitude-proxy-onset]]).

a.setBins(4)
osc(20, 0.1, 0.6)
  .kaleid(() => 2 + Math.pow(a.fft[3], 2) * 10)   // x² → quiet stays still, peaks jump
  .rotate(() => a.fft[2] * 3)
  .out(o0)

4 — a bass pump (sidechain-style duck). Borrow the sidechain move from music: let bass amplitude pull brightness down with an exponential curve, so the whole image ducks on the kick and breathes back — a visual pump locked to the low end ([[amplitude-pump-proxy-sidechain]]).

a.setBins(4)
osc(20, 0.1, 0.6)
  .brightness(() => -Math.pow(a.fft[0], 0.5) * 0.5)   // exp bass → pumping duck
  .out(o0)

5 — the safety rails. Live, the bridge must not pin or throw. FFT values can exceed 1, an out-of-range index reads undefined, and a pinned parameter looks broken. Guard every read (a.fft[i] || 0), clamp to 1, gate the floor with setCutoff, normalise peaks with setScale, and smooth so nothing strobes ([[fft-value-range-exceeds-one]], [[fft-index-out-of-bounds]], [[reactive-param-pinned-fails-sanity]], [[base-term-silence-floor]]).

a.setBins(4)
a.setSmooth(0.8)                             // no strobe
a.setScale(6)                                // normalise peaks toward 1
a.setCutoff(3)                               // gate the rumble floor
const bin = (i) => Math.min(a.fft[i] || 0, 1)  // guard out-of-bounds + clamp
osc(20, 0.1, 0.6)
  .brightness(() => 0.2 + bin(0) * 0.4)      // base term keeps it visible at silence
  .rotate(() => bin(2) * 3)
  .out(o0)

6 — the full bridge (the capstone). Everything together: guarded/clamped bins, all four bands coupled, a squared onset-pop on the highs, an exp bass-pump duck, tuned scale/smooth/cutoff, and a base term so the image lives even in silence ([[four-band-layout-shared-surface]], [[smoothing-global-per-value-workaround]]):

a.setBins(4)
a.setSmooth(0.82)
a.setScale(6)
a.setCutoff(3)
const b = (i) => Math.min(a.fft[i] || 0, 1)             // guarded, clamped bins
osc(20, 0.1, () => 0.5 + b(1) * 0.4)                    // low-mid → detail
  .brightness(() => 0.15 - Math.pow(b(0), 0.5) * 0.35)   // bass → exp pump duck (with a floor)
  .rotate(() => b(2) * 3)                                // high-mid → rotation
  .modulate(noise(2), () => b(1) * 0.3)                  // low-mid → warp
  .kaleid(() => 2 + Math.pow(b(3), 2) * 8)               // highs → squared onset-pop symmetry
  .out(o0)

What good looks like. A trustworthy bridge never pins (no parameter stuck at min/max — that reads as frozen or blown-out) and every band earns its coupling: you can tag exactly which Strudel voice drives which visual move. The tell of a fragile bridge is a parameter that clips to the rail the moment the drop hits, or a NaN that blacks the screen from an unguarded read. Tune setScale/setSmooth until each parameter swings through its middle range on a real track. (Skill map: live-visualist Domain C — audio-reactivity that feels musical, made robust enough to perform.)

Now make it yours. Tag your kick, hats, and pad Strudel voices and confirm each drives its intended band. Try a 1 - x² transfer for an inverse pop. Add per-value smoothing inside one thunk (a running lerp) where the global setSmooth is too blunt. Route the highs to .pixelate instead of .kaleid and compare. Add a stable, un-reactive layer via .layer(...) so the reactive bands have contrast.