Diffusion models are attractive because they are simultaneously analytically tractable and flexible

Generative modelling faces a tension between two usually-conflicting goals. Tractable models (e.g. a single Gaussian or Laplace) can be analytically evaluated and cheaply fit to data, but cannot capture the rich structure of complex datasets. Flexible models can fit arbitrary structure, but evaluating, training, or sampling from them is typically expensive or unstable. Diffusion models resolve this tension: because each forward step adds only a small amount of Gaussian noise, every reverse step is approximately Gaussian and the training objective decomposes into closed-form terms (tractable), yet the reverse-step means are produced by an unconstrained neural network that can model any data distribution (flexible). This is a core reason diffusion became dominant in generative image and audio work. The unavoidable price is the long Markov chain: generation is slow compared to single-pass generators like GANs.

Examples

A plain Gaussian is fully tractable but cannot generate a photorealistic face; a GAN is flexible but trains unstably. Diffusion keeps a Gaussian-based, closed-form training objective while letting a deep U-Net supply the flexibility.

Assessment

State the tractability-vs-flexibility tension in generative modelling and explain, in terms of the small-noise-step assumption, how diffusion models achieve both at once.