The diffusion training objective decomposes into per-timestep KL divergence terms plus a reconstruction term

Minimising the negative log-likelihood of a diffusion model is intractable directly, so training uses a variational lower bound (VLB) which upper-bounds the NLL. The VLB splits into: L_T (KL between the noised distribution at T and a standard Gaussian prior, constant since q has no learnable parameters), L_{t-1} for t=2..T (KL between the true reverse posterior conditioned on x_0 and the approximate reverse p_theta), and L_0 (a reconstruction term for the final denoising step). The L_{t-1} terms compare two Gaussian distributions at each timestep and can be computed in closed form. Optimising the VLB is equivalent to minimising all KL terms across all timesteps, which corresponds to teaching the network to denoise at every noise level simultaneously.

Examples

L_T is ignored during training since q has no parameters. The dominant training signal comes from the L_{t-1} terms, which motivate the noise-prediction formulation.

Assessment

Sketch the VLB decomposition: what is L_T, what are the L_{t-1} terms, and which can be computed analytically? Which terms are actually optimised during training?