Interactive music systems react to performer input in real time, ranging from deterministic score-following to autonomous improvising agents

Interactive music systems span a wide design space: at one end, score-following systems track a known musical plan and cue pre-composed material; at the other, fully autonomous machine-listening agents improvise freely without a plan. Intermediate designs include rule-based response systems, machine learning systems trained on a performer’s history, and Markov chain improvisers that build up probabilistic models from incoming MIDI data in real time. A recurring design question is the directness of control: how transparently does the performer’s action translate to musical result. Low-latency audio analysis is the perceptual glue; systems operating with more than 20 ms latency will feel detached.

Examples

George Lewis’s Voyager (1985) is an autonomous agent improviser. Max/MSP and SuperCollider are common platforms. JamFactory (1987) used Markov chains trained on incoming MIDI.

Assessment

What distinguishes score-following from free machine listening in interactive music? Why is latency a critical constraint in interactive music systems?