home/ atoms/ diffusion-closed-form-noising

The reparameterization trick lets you sample a noisy x_t at any timestep t directly without simulating the full chain

A key mathematical convenience is that the marginal distribution q(x_t|x_0) can be computed in closed form using cumulative noise products: x_t = sqrt(alpha_bar_t)*x_0 + sqrt(1-alpha_bar_t)*epsilon where epsilon ~ N(0,I) and alpha_bar_t is the product of all alpha_s = (1-beta_s) for s=1 to t. This means training does not require simulating t denoising steps for each training sample; you can directly corrupt a clean image to any noise level in one shot. This is why diffusion model training is efficient: sample a random t uniformly, compute x_t in one step using the reparameterization, predict the noise, compute loss. Without this property, training would require simulating the entire forward chain for every sample.

Examples

Training: draw x_0 from dataset, sample t uniformly from [1,T], sample epsilon ~ N(0,I), compute x_t = sqrt(alpha_bar_t)*x_0 + sqrt(1-alpha_bar_t)*epsilon, predict epsilon with the network, minimize MSE.

Assessment

Without simulating the chain step-by-step, how do you compute x_t for a given t, x_0, and sampled noise? What formula is used and why is this possible?

“A nice property of the above process is that we can sample”
corpus · what-are-diffusion-models-lil-log-lilian-weng · chunk 2