home/ atoms/ rave-nn-tilde-latent-manipulation

Splitting RAVE encode and decode in nn~ lets performers process individual latent dimensions live

When the encode and decode methods are used separately in Max/MSP (rather than the forward shortcut), the N latent signals appear as audio-rate outlets, and Max/Pd signal-processing tools can operate on them directly. Because variational RAVE latent values are sampled from an approximately isotropic normal distribution, they typically stay within a few standard deviations of zero. A performer can route specific latent dimensions to sliders for manual control, add LFO modulation to induce timbral movement, silence or randomize specific dimensions, or cross-fade between two encoded sources in latent space to create hybrid timbres. These manipulations turn the model into a live instrument.

Examples

Max patch: nn~ model encode 2048 → route one latent outlet through a *~ 0. to silence that dimension → nn~ model decode 2048, changing the timbre.

Assessment

A performer wants their RAVE model to slowly morph between two timbres over 16 bars. Describe a Max patch strategy using encode and decode separately.

“Having an explicit access to the latent representation yielded by RAVE allows us to interact with the representation using Max/MSP or PureData signal processing tools”
corpus · rave-realtime-audio-variational-autoencoder-train-your-own-n · chunk 2