A cosine noise schedule gives gentler early-stage transitions than the linear schedule, improving likelihood

In the original DDPM, noise variances beta_t increase linearly from 1e-4 to 0.02. Nichol and Dhariwal (2021) found this destroys the image too quickly at small t, wasting the high-signal early steps and hurting log-likelihood. The cosine schedule sets beta_t from a cosine-based function that gives a near-linear drop in the middle of the process but only subtle changes near t=0 and t=T, so more of the noising happens in the middle steps. This gives the network more useful training signal at low noise levels (where structure remains) and less wasted capacity at very high noise. The schedule is clipped (e.g. at 0.999) to prevent numerical issues near t=T. It improved NLL over the linear schedule at the cost of slightly more complex schedule code.

Examples

Under the cosine schedule an image degrades more slowly in early steps than under the linear schedule, giving the network more structured training signal at low t.

Assessment

Why does the linear noise schedule cause problems at small t? Explain qualitatively how the cosine schedule addresses this.