Text conditioning is injected into the UNet by attention layers placed between ResNet blocks

To make the noise predictor follow the prompt (the paper’s ‘text conditioning’), attention layers are inserted between the UNet’s ResNet blocks. The ResNet blocks do not look at the text directly; the attention layers merge the text token representations into the latents, so the following ResNet block can use that incorporated text information as it processes the array. This is how the prompt steers what image is denoised into existence — without it the same UNet still produces good images but with no control over their content. Placing conditioning at multiple layers (not just once at the input) lets text influence processing throughout the network’s depth.

Examples

UNet: ResNet block → attention layer (merges text tokens into latents) → ResNet block → attention → … Remove the attention layers and the model generates images but ignores the prompt entirely.

Assessment

Explain how attention layers between ResNet blocks incorporate text into the UNet, and why a text-conditioned UNet can follow a prompt while a plain diffusion UNet cannot control image content.