ComfyUI's smart memory manager offloads models to CPU when GPU VRAM is insufficient

ComfyUI tracks every loaded model in a global list (current_loaded_models) and assigns each a VRAMState (NO_VRAM, LOW_VRAM, NORMAL_VRAM, HIGH_VRAM). Before running inference, load_models_gpu() estimates required VRAM, evicts least-recently-used models to CPU, and partially loads the current model — only the layers needed for the current forward pass are on GPU at once. This enables running large models on GPUs with as little as 1 GB VRAM. —lowvram, —novram, and —highvram flags override the automatic selection. The misconception is that ComfyUI requires a high-end GPU; the system degrades gracefully to CPU execution via —cpu.

Examples

On a 4 GB GPU: UNet is partially loaded, VAE decodes in CPU, then VAE is swapped back. With —highvram, all loaded models stay on GPU permanently.

Assessment

Explain the trade-off between —highvram and —lowvram modes. When would you use —novram and what is the performance cost?