Composing Audio-Visual Sets From Musical State and Cues

This module is where OSC plumbing becomes authorship. The whole task is a short AV set section — the kind you would play in a club or gallery AV night with Ableton Live as the musical brain and an OSC-capable visual host (CoGe, TouchDesigner, a browser sketch) as the projector feed. What separates a composed section from a screensaver is that the visuals know the music’s structure: which clip fired, where the bar falls, when the section’s tempo shifts. The central idea — a DAW’s internal musical state, exposed over OSC, is a richer driver than audio analysis — reframes Live as a choreography server, with audio-reactivity kept as texture, not backbone.

The arc starts supported: first, draw a clip envelope that emits discrete visual cues and watch triggers land with sample-accurate, clip-bound timing (the automation-as-visual-trigger procedure is your JIT how-to). Next, subscribe to per-track output meters so the kick and bass tracks lend continuous energy to intensity and colour. Then couple rhythm itself: stretch a video clip to bar length so visual loops peak with musical phrases, and give a scene its own tempo and signature so firing it retunes the whole system over OSC. Only then do you take the supports away and compose the capstone unsupervised, layering two live sources into a composited “third space” that reads as one coherent place. Note that “reads as one coherent whole” is not a separately taught skill: it is the capstone-level synthesis, evidenced by integrating the individually taught drivers into one section rather than by any single atom.

The required atoms each gate the capstone directly — remove any one and a named capstone element (cue triggers, meter energy, bar-stretch, scene tempo change, the composite) fails. Supporting atoms enrich: the beat-event listener offers an alternative clock hook for beat-locked flourishes, and the DSP-mapping concept deepens the audio-reactive texture layer you blend on top. Drill the envelope-cue and meter-mapping moves until automatic — they recur constantly mid-performance.

Runnable examples

Generated from the context/ instrument corpus by concept (redistributable idioms only). Do not edit — regenerate with gen-module-examples.mjs.

audio-reactive-map

updateAudio(); scale(1 + amp * 0.01)

_{p5live-0044 · CC0-1.0}

voronoi(() => 4 + a.fft[0] * 10).out()

_{hydra-0035 · CC0-1.0}

spectral-band-split

[ihi, imid, ilo] >> rgb

_{punctual-0041 · CC0-1.0}