home/ modules/ generative-av-mapping-across-tools

Generative and Multi-Tool Audio-Visual Mapping

  • learner can split audio into named frequency bands and bind each band to a different visual parameter in a VJ tool
  • learner can drive visuals with a probabilistic (non one-to-one) rule set fed by audio features
  • learner can choose an appropriate sync granularity — from fine per-parameter to coarse cue-passing — for a given AV role, including performer-anchored setups

Build a generative audio-reactive scene that band-splits incoming audio to separate visual parameters and uses at least one probabilistic feature-to-visual rule (not a direct map); document where you sit on the sync-granularity spectrum and sketch how you'd anchor the visuals to a moving performer.

This module is where audio-reactive visuals stop being a novelty and start being a set. In a real club or festival slot — you as the visualist beside a live electronic act, or driving both sound and image solo — a single amplitude-to-brightness map reads as mechanical within a minute. The whole task here is a scene that listens like a musician: kick, snare, lows and mids each own a different visual parameter, and at least one visual behaviour is chosen probabilistically, biased by the music rather than chained to it.

The arc starts supported: in your VJ tool of choice, follow the band-splitting procedure (drag named analysis bands onto parameters, one band per visual job) until routing a kick to a pulse and a low swell to a slow bloom is a reflex — this binding move is the part-task drill. Next, replace one direct map with a rule inspired by the probabilistic feature-to-visual principle: let a feature weight a random choice among visual actions, so the audio biases the distribution and chance supplies the surprise. Finally, zoom out to the performance frame: the sync-granularity spectrum tells you whether your role wants per-parameter firehose or coarse cue-passing, and the performer-anchored concept shows how audio plus positional tracking pins visuals to a moving body — both feed the written half of the capstone.

The four required atoms gate the capstone directly: no band-splitting, no scene; no probabilistic rule, no generative claim; no granularity or anchoring understanding, no credible documentation. The supporting atoms enrich the “across tools” view — Gibber’s assignment-plus-scaling idiom and TouchDesigner’s any-channel-drives-any-parameter insight show the same mapping discipline transposed, useful when your rig changes but not prerequisites for this build.

Atoms in this module

Required — these gate the capstone

Splitting incoming audio into named bands lets a VJ hook each band to a different visual parameter
Procedure L3 Craft JI
Analysing sound features (frequency, tempo, gesture) as inputs to probabilistic rules generates visuals that respond to music without one-to-one mapping
Principle L3 Craft JI
AV sync ranges from fine-grained per-parameter control for soloists to coarse cue-passing for visualist collaborators
Principle L3 Craft JM
A performer's live audio plus tracked position can drive visuals anchored to their body in real time
Concept L3 Craft JH

Supporting — enrichment, not gating

Gibber maps audio objects directly to visual properties using bias and scalar for range control
Procedure L3 Craft JH
In TouchDesigner any CHOP channel can drive any operator parameter, so audio, MIDI and beat inputs are interchangeable
Concept L2 First instrument JI