home/ modules/ deriving-the-diffusion-model-from-scratch

Deriving and Training a Diffusion Model from Scratch

L2First instrument 6h K · AI & real-time generative AV 12 atoms

Learning objectives

learner can explain the forward noising and reverse denoising processes and the noise-prediction objective
learner can implement the DDPM training loop: q_sample, timestep sampling, U-Net with time conditioning, MSE on noise
learner can run the reverse sampling loop and apply the input normalization and augmentation that make it work

Capstone — one whole task that evidences the objectives

Implement a working DDPM on a small image dataset: build the U-Net denoiser with sinusoidal timestep conditioning and FiLM ResNet blocks, implement q_sample via the nice property, train with the simplified MSE-on-noise loss over randomly sampled timesteps, and generate samples by running the reverse loop — with [-1,1] normalization and horizontal-flip augmentation.

Prerequisite modules

Orienting to Generative AI for Live AV

Every generative-visual tool a live-AV performer might lean on — Stable Diffusion textures for projection, real-time img2img reactive layers, custom models trained on a crew’s visual identity — sits on one algorithm: DDPM. This module has you build that algorithm end to end on a small image dataset, so that when you later fine-tune, distill, or bend a diffusion model for a set, you know exactly which knob you’re turning and why a “steps” slider or noise schedule changes what hits the screen.

The arc starts on paper: the fixed forward noising process and the learned reverse process, then the self-supervised trick of training a network to predict known added noise. With the concepts in place, you build in supported steps — the variance schedule and its precomputed alpha products first, then the one-shot corruption function (“q_sample implements the ‘nice property’”) as your first runnable exercise. Next comes the denoiser itself (“DDPM uses a U-Net with skip connections… taking noisy image and timestep as input”), wired into the training loop with random per-example timesteps and MSE on noise. The unsupported capstone closes the loop: train the model with [-1,1] normalization and horizontal flips, then generate images via the reverse sampling procedure.

Every required atom gates that capstone — you cannot write q_sample without the schedule, condition the U-Net without understanding what it predicts, or get sane samples without normalization. The supporting atoms enrich rather than gate: seeing how the same inference loop appears in production SD pipelines, why the model learns a distribution rather than memorizing images, and how latent diffusion compresses this exact process to run at performance-friendly cost. Drill q_sample and the reverse loop until they’re reflexive; they recur in every diffusion project you’ll touch.