Generating Audio from Latent and Spectral Representations

This module closes the loop that every spectrogram-based neural instrument depends on: getting sound back out. In a live-coding or studio rig built around a trained VAE — say, a model trained on a folder of found percussion or field recordings — the decoder hands you a normalized log-magnitude image, not a waveform. Until you can invert that image reliably, the model is mute. The whole task here is a small but complete generator: pull a point from the latent prior for a genuinely new sound, reconstruct an encoded input for an A/B comparison, and turn both decoder outputs into audible files you could drop into a sampler or a set.

The arc starts supported: first internalize the three-stage encode–decode–invert pipeline as a map, then work through the inversion procedure step by step — strip the channel dimension, denormalize with the stored per-sample min/max, de-log back to linear amplitude, then iSTFT with the matching hop length. That inversion chain is the part-task drill: it recurs on every sound you ever generate, and skipping or reordering a step silently wrecks amplitudes, so it must become automatic. From there the two generation modes (reconstruction vs. prior sampling) are exercised side by side, and the Griffin-Lim concept supplies the language for the write-up: the network discarded phase, Griffin-Lim converges on a plausible-but-wrong one, and that is exactly the metallic texture you will hear.

The four required atoms gate the capstone directly — without any one of them you either cannot produce audio or cannot explain what you hear. The supporting atoms enrich the picture: RAVE’s architecture and its reparametrized encoder show where these latents come from in a realtime instrument, and the segment-processing principle previews the memory tradeoffs waiting when models scale up.