RAVE is a variational autoencoder that encodes audio into a compact latent space and decodes it back in realtime

RAVE (Realtime Audio Variational autoEncoder), like vschaos2, is an autoencoder: it takes sound in, produces sound out, and is trained to reconstruct the sounds of its dataset. Processing has two stages: an encoder compresses a window of incoming audio (e.g. 2048 samples) into a low-dimensional set of latent variables (commonly 128); a decoder inverts those latents back into audio. The forward pass is simply these two chained. Because the bottleneck is variational, latent space can be smoothly sampled and interpolated, giving continuous control over timbre — the latent dimensions become synthesis parameters shaped by whatever corpus you trained on, effectively learning a controllable instrument. In the nn~ Max/PD object the encode method exposes each latent dimension as a separate outlet and decode takes latent values at its inlets; wiring every encode outlet straight into decode reproduces the forward transform, while splitting the wires lets you read or overwrite individual latents in between. RAVE exposes only a controllable subspace of latents (per the latent-space morphology) whereas vschaos2 exposes them all. The trained model runs in realtime, exportable as a TorchScript file for Max/MSP, Pure Data, or the RAVE VST.

Examples

Workflow: rave preprocess on a folder of recordings, rave train --config v2, rave export --streaming to get a .ts file loadable in nn~ or the RAVE VST. In nn~: encode → N latent outlets → decode inlets reproduces forward; split the wires to read or overwrite individual latents.

Assessment

Explain the three stages of the RAVE workflow and what the latent space represents; describe what changes when you move a single latent dimension in the decoder. State the difference between using forward versus separate encode/decode in nn~, and what chaining every encode outlet directly into decode reproduces.