Generative and Multi-Tool Audio-Visual Mapping
Learning objectives
- learner can split audio into named frequency bands and bind each band to a different visual parameter in a VJ tool
- learner can drive visuals with a probabilistic (non one-to-one) rule set fed by audio features
- learner can choose an appropriate sync granularity — from fine per-parameter to coarse cue-passing — for a given AV role, including performer-anchored setups
Capstone — one whole task that evidences the objectives
Build a generative audio-reactive scene that band-splits incoming audio to separate visual parameters and uses at least one probabilistic feature-to-visual rule (not a direct map); document where you sit on the sync-granularity spectrum and sketch how you'd anchor the visuals to a moving performer.
Prerequisite modules
This module is where audio-reactive visuals stop being a novelty and start being a set. In a real club or festival slot — you as the visualist beside a live electronic act, or driving both sound and image solo — a single amplitude-to-brightness map reads as mechanical within a minute. The whole task here is a scene that listens like a musician: kick, snare, lows and mids each own a different visual parameter, and at least one visual behaviour is chosen probabilistically, biased by the music rather than chained to it.
The arc starts supported: in your VJ tool of choice, follow the band-splitting procedure (drag named analysis bands onto parameters, one band per visual job) until routing a kick to a pulse and a low swell to a slow bloom is a reflex — this binding move is the part-task drill. Next, replace one direct map with a rule inspired by the probabilistic feature-to-visual principle: let a feature weight a random choice among visual actions, so the audio biases the distribution and chance supplies the surprise. Finally, zoom out to the performance frame: the sync-granularity spectrum tells you whether your role wants per-parameter firehose or coarse cue-passing, and the performer-anchored concept shows how audio plus positional tracking pins visuals to a moving body — both feed the written half of the capstone.
The four required atoms gate the capstone directly: no band-splitting, no scene; no probabilistic rule, no generative claim; no granularity or anchoring understanding, no credible documentation. The supporting atoms enrich the “across tools” view — Gibber’s assignment-plus-scaling idiom and TouchDesigner’s any-channel-drives-any-parameter insight show the same mapping discipline transposed, useful when your rig changes but not prerequisites for this build.
Atoms in this module
Required — these gate the capstone
Supporting — enrichment, not gating
Part of curricula
- Audio-Visual Performer — integrated, synced live AV — Compose the whole (generative & AI-layered AV) required
- Live Visualist — zero to performing live-coded & generative visuals — Perform the set — live-coded, generative, audio-reactive visuals for an audience required
- VJ — visual performance with projection, light & video — Lock to the music: sync to a DJ or band recommended
Unlocks — modules that require this one