Progressive distillation halves the required sampling steps by training a student to match two teacher steps in one

Progressive distillation (Salimans and Ho 2022) accelerates deterministic diffusion samplers by iterative knowledge distillation. A student model is initialised from the teacher and trained so that one student step matches two teacher DDIM steps, denoising toward that two-step target instead of toward the original sample x_0. After one round, the student generates in T/2 steps at the same quality as the teacher at T steps. The process repeats, each round halving the step count (e.g. from T=1024 toward a handful of steps). It only works for deterministic samplers (eta=0 DDIM) because the two-teacher-step target must be deterministic. It requires no new training data, only the pre-trained teacher, making it cheap to run repeatedly.

Examples

Starting with a 1024-step teacher: repeated rounds of progressive distillation shrink the student toward very few steps at comparable quality, enabling near-real-time sampling on consumer hardware.

Assessment

Describe the progressive-distillation training target. Why can each round halve the step count? What requirement must the teacher model meet?