DDSP makes DSP components differentiable so neural networks can drive classical synthesizers

DDSP (Differentiable Digital Signal Processing) wraps physically interpretable signal processors — oscillators, filters, reverb — in TensorFlow operations so that gradients can flow through them during training. A neural network predicts synthesizer parameters (fundamental frequency, harmonic amplitudes, filter magnitudes) and the DSP layer converts those parameters into audio. Because the synthesis is differentiable, standard gradient-descent optimizers can train the entire stack end-to-end using audio reconstruction losses. This contrasts with fully neural vocoders (WaveNet, GANs): DDSP retains explicit DSP structure, which gives better sample efficiency (~10 min of audio per instrument), easier interpretability, and stable pitch control. A common misconception is that differentiable synthesis requires replacing DSP with neural operations — in DDSP, the DSP code stays; only its parameters come from the network.

Examples

import ddsp
outputs = network(inputs)
harmonic = ddsp.synths.Harmonic()
audio = harmonic(outputs['amplitudes'], outputs['harmonic_distribution'], outputs['f0_hz'])

Assessment

Explain why DDSP needs less training data than a purely neural vocoder. Describe the flow from raw audio to synthesizer parameters to audio reconstruction.