img2img and txt2img mode in StreamDiffusion differ in their latent initialization and CFG constraints

In img2img mode, each call encodes the input image into latent space with added noise at the earliest timestep in t_index_list, then denoises it toward a prompt-conditioned output. The source image anchors the output’s structure. In txt2img mode, latents are initialized from pure Gaussian noise—no image input is needed. StreamDiffusion enforces two constraints: (1) txt2img mode accepts only cfg_type='none' (the residual CFG modes require an image to seed the stock noise); (2) denoising batch with frame_buffer_size > 1 is only supported for txt2img with SD-Turbo. The wrapper validates these at construction time and raises ValueError on violations.

Examples

# img2img: requires an image input
stream_i2i = StreamDiffusionWrapper(..., mode='img2img', cfg_type='self')
result = stream_i2i(image=my_frame, prompt='oil painting')

# txt2img: no image input, cfg_type must be 'none'
stream_t2i = StreamDiffusionWrapper(..., mode='txt2img', cfg_type='none')
result = stream_t2i.txt2img()

Assessment

Why does txt2img mode reject cfg_type='self'? Trace through the RCFG logic in unet_step to explain what would go wrong if you tried.