MIR uses content-based audio feature extraction and machine learning to automate musical analysis and similarity search

Music Information Retrieval (MIR) is the field of automatically extracting meaningful information from musical signals or symbolic scores. Content-based MIR systems use automated analysis to establish descriptors from audio rather than relying on metadata alone. The pipeline typically involves computing low-level acoustic features from audio frames (spectral centroid, MFCCs, chroma, onset strength, tempo), combining frame-level features into summary statistics or higher-level descriptors, and applying machine learning to map features to musical categories (genre, mood, key, instrument, similarity). A central challenge is annotation: musical labels are subjective, experts disagree, and large clean datasets are expensive to create.

Examples

Shazam identifies songs by fingerprinting spectral peaks. Spotify recommendation uses embeddings trained on audio features and listening history. Melodyne uses pitch detection (a MIR sub-problem) for pitch editing.

Assessment

What is the typical pipeline from raw audio to a music genre classification label? Name two challenges in creating ground-truth annotations for MIR tasks.