Stable Diffusion runs in two modes: text-to-image and image-plus-text (img2img)

Stable Diffusion is versatile and used in two primary ways. In text-to-image (text2img) the only input is a text prompt and the model generates an image from scratch. In its other main mode the input is text plus an image, and the model alters the supplied image according to the prompt rather than starting from nothing. Knowing the model accepts both a prompt-only and a prompt+image input is the basic orientation for using SD as a creative tool; the article introduces img2img at a high level as ‘making it alter images’ without detailing the internal mechanism.

Examples

text2img: prompt ‘a red rose in impressionist style’ → a new image. img2img: supply a rose photo + prompt ‘impressionist style’ → the model alters that photo toward the prompt.

Assessment

State the two main ways Stable Diffusion can be used and the inputs each requires. Which mode takes an existing image as input in addition to text?