RAVE's variational encoder reparametrizes latent samples using the mean and log-variance trick

The variational encoder outputs 2×latent_size channels: the first half is the mean (μ), the second is a scale (σ) after a softplus nonlinearity. The reparametrization trick samples z = μ + σ·ε where ε ~ N(0,1), enabling gradients to flow through the sampling step. A KL-divergence regularization term (μ² + σ² − log σ² − 1) is summed and weighted by beta — a beta-VAE formulation. In phase 2 (adversarial), the encoder is frozen (set_warmed_up) so the GAN gradients cannot destroy the structured latent space established in phase 1.

Examples

In the variational encoder: mean, scale = z.chunk(2, 1); std = softplus(scale)+1e-4; z = randn_like(mean)*std + mean; kl = (mean*mean + var - logvar - 1).sum(1).mean().

Assessment

Why does RAVE freeze the encoder during phase 2? What would happen to the latent space structure if GAN gradients could update the encoder weights?