Every KL term in the diffusion VLB compares two Gaussians and is therefore analytically tractable

A key tractability property of diffusion models is that the true reverse posterior q(x_{t-1}|x_t,x_0), when conditioned on the clean x_0, is a Gaussian with a known closed-form mean and variance. Since the model approximation p_theta(x_{t-1}|x_t) is also parameterised as a Gaussian, every KL divergence term L_{t-1} in the VLB reduces to a closed-form expression involving only means and variances — no Monte Carlo estimation is needed. This is in contrast to VAEs, where the posterior is approximated by an encoder network. The exceptions are L_0 (the final-step reconstruction term), which Ho et al. handle with a separate discrete decoder, and L_T, which is constant and ignored. This analytical tractability is a major reason diffusion models train more stably than GANs.

Examples

The KL divergence between two multivariate Gaussians has a closed-form formula, enabling exact gradient computation for each timestep without sampling.

Assessment

Why is the KL term L_{t-1} tractable in diffusion models when it is not in many other latent-variable models? What property of the forward process makes this possible?