The Text-Conditioned Image Generation Stack

When you drive generative visuals in a live set — prompting imagery that tracks the music while a dancer’s pose or a projected silhouette constrains its composition — you are choosing between real architectures with real trade-offs. This module builds the mental map that makes those choices deliberate: how text becomes an image, where each control knob lives in the stack, and which conditioning strategy fits a prompt-and-pose-controlled performance visual.

The arc starts supported. First orient with the whole system: “Stable Diffusion is a pipeline of three neural networks” gives you the skeleton (text encoder, U-Net + scheduler, decoder), and the text2img/img2img distinction tells you what inputs the pipeline accepts. From there, descend one level at a time as just-in-time how-to: “CLIP jointly trains text and image encoders” explains why a text prompt can steer pixels at all, and the U-Net backbone and “attention layers between ResNet blocks” atoms show exactly where that steering enters the generator. With the base stack traced, the three alternative conditioning strategies — unCLIP’s hierarchical prior-then-decoder, Imagen’s frozen language model, ControlNet’s zero-convolution spatial branch — become comparable rather than a blur of paper names. The capstone then removes the scaffolding: you narrate the full stack unaided and commit to an explicit pick for your own rig, with a verdict on each candidate.

Every required atom gates that capstone — miss the U-Net’s structure and you can’t place the attention layers; miss any one of the three conditioning papers and the comparison collapses. The supporting atoms enrich the picture: latent diffusion explains why this runs on performance-grade GPUs at all, GAN latent arithmetic gives historical contrast for controllable imagery, and the attention/normalization/xformers atoms deepen the engineering view without being needed to draw the map.