A VAE generates sound by encoding a spectrogram to a latent point, decoding it, and inverting the STFT

A Variational Autoencoder (VAE) can generate audio by operating on spectrograms rather than raw waveforms. The pipeline has three stages: (1) Encode — pass a log-magnitude spectrogram through the encoder to produce a point in the latent space. (2) Decode — feed that latent point through the decoder to produce a reconstructed spectrogram. (3) Convert — apply the inverse Short-Time Fourier Transform (iSTFT) to recover a waveform. Because VAEs operate on magnitude spectrograms, phase is not stored; the iSTFT step must reconstruct phase (here via Griffin-Lim). The result is recognisable but imperfect. Understanding this three-stage pipeline is prerequisite to using or modifying any neural audio generator built on spectrograms.

Examples

A VAE trained on spoken-digit spectrograms: passing the spectrogram for digit ‘3’ through encode→decode→iSTFT produces a recognisable but slightly distorted ‘3’. Latent-space sampling (skipping the encoder) produces novel digits.

Assessment

Draw the signal flow from an input audio file to a VAE-generated output audio file. At which stage is phase information not carried through, and which algorithm reconstructs it?