RAVE dataset quality needs a balance between homogeneity and diversity

RAVE’s latent space must compress a corpus into a compact, navigable representation. If the dataset comprises very different types of sounds, the model may struggle to learn everything and to gather everything, and the latent space becomes cluttered and hard to control. If the dataset is too similar, the model may fall into a low-capacity behavior with very low variety, under-using its latent dimensions. The practical sweet spot is a single sound source with natural variation — one instrument across its range, one voice in varied speech, one type of percussion in different rhythms. Normalizing loudness across recordings prevents the model from using amplitude as a dominant latent axis.

Examples

Good: several hours of solo cello recordings. Poor: 30 min drums + 30 min vocals + 30 min piano mixed together, which the model struggles to gather into one coherent latent space.

Assessment

A user trains RAVE on a mix of field recordings, synthesizers, and spoken word and finds the latent space feels chaotic. Diagnose the issue and prescribe a dataset curation strategy.