Mapping Audio Features to Visual Parameters

This module builds toward the bread-and-butter task of the audiovisual live-coder: making a visual rig breathe with the music coming out of your DAW. In a club or gallery set, nobody hands you a clean analysis feed — your techno kick, pad swells, and hat patterns leave the DAW as an output stream, and the visuals only react if you close the loop yourself and then choose mappings that read as musical rather than merely synchronized.

The arc starts fully supported. First, get signal flowing: the loopback-routing procedure (“route system audio back in as an input via a platform-specific loopback device”) is your JIT how-to for the single most common setup failure, whether you’re on PulseAudio monitors, BlackHole, or a virtual cable. With audio arriving, the two-stage extract-then-map model (“extract DSP data from audio and map it to visual parameters”) frames your first exercise: drive one obvious parameter — size from loudness — and feel how curve and scaling choices change the result. Then generalize: the insight that any channel can drive any parameter (“any CHOP channel can drive any operator parameter”) turns reactivity into a wiring-and-remapping habit, which is why that mapping move is the part-task drill — you’ll repoint sources to destinations dozens of times per set. Note that these ideas were first articulated in different tools (p5.js for extract-then-map, TouchDesigner for channel interchangeability), but they are the same model wearing different clothes: apply them in whichever visual environment your patch lives in. Finally, the eye-versus-ear asymmetry concept gives you the judgment to notice when a mapping is technically correct but perceptually wrong, exactly the repair the capstone demands.

Every required atom gates the capstone directly: without routing there is no signal, without the mapping model and channel interchangeability there are no three distinct mappings, and without the perceptual lens you cannot diagnose the out-of-step moment. There are no supporting atoms here — this module is all load-bearing.

Runnable examples

Generated from the context/ instrument corpus by concept (redistributable idioms only). Do not edit — regenerate with gen-module-examples.mjs.

audio-reactive-map

updateAudio(); scale(1 + amp * 0.01)

_{p5live-0044 · CC0-1.0}

voronoi(() => 4 + a.fft[0] * 10).out()

_{hydra-0035 · CC0-1.0}

modulation-warp

osc(4).modulate(src(o0), 0.6).out(o0)

_{hydra-0022 · CC0-1.0}

s0.initP5(); src(s0).modulate(noize(), 0.3).out()

_{p5live-0038 · CC0-1.0}

feedback-trail

osc(4).modulate(src(o0), 0.6).out(o0)

_{hydra-0022 · CC0-1.0}

function draw(){ fill(0, 20); rect(0, 0, width, height); circle(mouseX, mouseY, 40) }

_{p5live-0003 · CC0-1.0}

scale-pulse

uv *= 1.0 + 0.3 * sin(u_time * 2.0);

_{glsl-0030 · public-domain}

updateAudio(); scale(1 + amp * 0.01)

_{p5live-0044 · CC0-1.0}

grain-glitch

col += (h21(st + fract(u_time)) - 0.5) * 0.15;

_{glsl-0026 · public-domain}

flow-field

let ang = noise(x*0.01, y*0.01) * TWO_PI

_{p5live-0008 · CC0-1.0}

spectral-band-split

[ihi, imid, ilo] >> rgb

_{punctual-0041 · CC0-1.0}

chromatic-aberration

col = vec3(texture2D(u_tex0,st+vec2(.005,0)).r, texture2D(u_tex0,st).g, texture2D(u_tex0,st-vec2(.005,0)).b);

_{glsl-0036 · public-domain}

visual-pulse

zoom (1 ~~ 2 $ osc 0.25) (circle 0 0.3) >> add

_{punctual-0022 · CC0-1.0}