Building Differentiable DSP Instruments with DDSP

This module builds toward designing your own neural instrument: a DDSP model you could train on ten minutes of a single sound source — a violin, your own voice, a field recording — and then play live with pitch and loudness curves from a controller or a live-coding pattern. Unlike black-box neural audio, a DDSP instrument stays legible on stage: every knob the network turns is a real synthesizer parameter you can inspect, override, or hijack mid-set.

The arc starts with the big idea — DSP components wrapped as differentiable operations so gradients flow through classical synthesis — before anything is wired up. From there you meet the Processor API, whose inputs-vs-controls split is the mental model everything else hangs on: raw network tensors in, physically valid synth controls out. With that frame, each processor becomes a supported exercise: the Harmonic synthesizer’s amplitude/distribution/f0 triple, the FilteredNoise synthesizer’s noise-shaping magnitude envelope, and the trainable Reverb’s learned impulse response. When amplitudes need to be strictly positive, the exp_sigmoid scaling atom is your just-in-time pointer; when you turn to data, the CREPE pitch-extraction and TFRecord preparation atoms walk you through the offline conditioning pipeline, with A-weighted loudness explaining why the model hears volume the way you do.

Every required atom is load-bearing for the capstone: you cannot chain the three processors, justify the amplitude mapping, or prepare the dataset without them. The supporting atom on zeroing partials above Nyquist enriches the picture — it explains why the harmonic stack never aliases — but the capstone stands without reciting it. By the end, the unsupported task is yours alone: assemble the full chain and account for what each stage contributes to the timbre.

Runnable examples

Generated from the context/ instrument corpus by concept (redistributable idioms only). Do not edit — regenerate with gen-module-examples.mjs.

reverb-space

s("cp").room(0.6).size(4)

_{strudel-0019 · CC0}

out: mix ~a ~b >> plate 0.3

_{glicol-0008 · MIT}