Dropping per-timestep loss weights and training with uniform MSE on noise prediction improves DDPM in practice

The theoretically derived loss L_t includes a per-timestep weighting factor depending on alpha_t. Ho et al. (2020) found empirically that simply training with unweighted MSE between predicted and true noise — ignoring the weighting — produces better sample quality. This simplified objective L_simple is an equally weighted average over all timesteps. The weighted version downweights terms at small t (low noise, where prediction is easiest) and upweights terms at large t; dropping it effectively upweights the low-noise regime, which may matter more perceptually. This is a practical empirical finding without a clean theoretical justification, but it is what all major diffusion implementations use.

Examples

The DDPM training algorithm uses L_simple rather than the full VLB: MSE on the noise prediction, uniformly averaged across all sampled timesteps.

Assessment

What is the difference between L_t and L_simple in DDPM training? Which performs better empirically, and what is traded off by using the simplified objective?