U-Net's symmetric downsampling-upsampling structure with skip connections makes it a standard diffusion backbone

U-Net consists of a downsampling (contracting) stack and an upsampling (expanding) stack. The downsampling path applies repeated 3x3 convolutions + ReLU + 2x2 max pooling; each level doubles the feature channels while halving spatial resolution. The upsampling path mirrors this, halving channels at each step. Skip (shortcut) connections concatenate feature maps from each downsampling level to the corresponding upsampling level, providing the high-resolution spatial detail that pooling would otherwise discard — essential for pixel-accurate output. For diffusion, the U-Net receives the noisy image (or latent), a timestep embedding, and optional conditioning, and predicts the noise. Attention layers are typically added at the lower-resolution stages where global context matters most.

Examples

In DDPM the U-Net adds group normalization and multi-head self-attention at low-resolution feature maps within the convolutional U-Net structure. LDM inserts cross-attention into the U-Net for text conditioning.

Assessment

Explain why skip connections are essential in U-Net for diffusion noise prediction. What information do they provide that the upsampling path alone cannot recover?