Some live coding systems fold machine learning and machine listening into the performance, letting the coder train and steer models live

Most live coding operates purely at the notational level: the performer writes symbols that an interpreter turns into sound or image. A newer strand embeds machine learning and machine listening directly into the live loop. The MIMIC research project (Chris Kiefer, Thor Magnusson, and collaborators) exemplifies this with its browser-based Sema language: it puts machine learning in the hands of creative coders and supports small-data training, real-time model initiation, and training done as part of the performance itself, rather than using only pretrained networks prepared in advance. Machine listening lets the system analyze incoming audio and respond, closing a feedback loop between sound and code. This raises a distinct performance question: what the audience sees projected may be the statistical outcome of training rather than legible instructions, complicating live coding’s ‘show us your screens’ transparency ethos.

Examples

In a Sema/MIMIC performance a coder records a few gestures or short audio examples, trains a small model live, then uses its output to drive synthesis parameters — the training run itself becomes visible performance material. Contrast a system that only calls a fixed, pretrained network as a black box.

Assessment

Explain how live in-performance training differs from using a pretrained model in a live coding set. Why does embedding machine learning complicate the ‘show us your screens’ transparency principle?