DDPM training samples a random timestep per example, corrupts it, and minimizes noise-prediction loss

The DDPM training algorithm is simple. Per iteration: (1) sample a real image x_0 from the dataset; (2) sample a timestep t uniformly from 1..T, independently for each image in the batch; (3) sample noise epsilon ~ N(0,I); (4) compute the noisy image x_t in one shot using the closed-form ‘nice property’ (rather than iterating the forward chain); (5) have the network predict the added noise from x_t and t; (6) backpropagate the noise-prediction loss and step the optimizer. Because a random timestep is drawn each iteration, the network sees all noise levels over training and the loss averages over them implicitly — this stochastic sampling is an unbiased estimator of the full ELBO objective, making it tractable without sweeping t=1..T every step. Concretely in PyTorch this is a U-Net plus an Adam optimizer looped over epochs and batches, commonly with a Huber loss; periodically the model is sampled to visualize and save progress.

Examples

t = torch.randint(0, timesteps, (batch_size,))       # one t per image
loss = p_losses(model, batch, t, loss_type='huber')
loss.backward(); optimizer.step()
# periodically: save_image(all_images, f'sample-{milestone}.png', nrow=6)

On Fashion-MNIST (28×28, batch 128), loss typically falls from ~0.46 to ~0.04 over ~6 epochs.

Assessment

Write the full DDPM training iteration in pseudocode. Why is random timestep sampling (rather than sweeping t=1..T each iteration) sufficient to optimize the full ELBO? What does a save_and_sample_every interval control, and why is it useful during training?