t-SNE embeds high-dimensional feature vectors in 2D so perceptually similar items cluster together

t-SNE (t-distributed Stochastic Neighbour Embedding) reduces a high-dimensional feature vector (a convnet activation for an image, or spectral/MFCC features for a sound) to two dimensions while preserving local neighbourhood structure: items close in feature space stay close in the 2D map. For artists this yields an explorable sound-map or image-map where proximity means perceptual similarity. ml4a provides guides for embedding both images and audio clips in 2D over their feature vectors, plus an openFrameworks viewer to navigate the map interactively. The common misconception is that distances BETWEEN clusters are meaningful — only within-cluster proximity is reliable; t-SNE distorts global distances, so a cluster being far from another says little.

Examples

Extract MFCC features from a sample library, run t-SNE to place each clip in 2D, and display them on a grid so clicking near a region plays texturally similar sounds.

Assessment

State what property t-SNE preserves and what it does NOT preserve, then explain why a t-SNE sound-map would group similar timbres but not necessarily order by pitch.