Concatenative synthesis drives resynthesis by selecting and joining database sound segments whose features best match a target

Concatenative synthesis builds a sound by selecting and joining units (segments of audio) from a pre-analysed database of sounds. Each unit is associated with a multi-dimensional feature vector. A target: either a specified feature trajectory or a live audio stream: drives a search for the best-matching database unit at each time step. The matching optimises two costs: target cost (proximity in feature space) and concatenative cost (smoothness of transitions between successive units). The audio-mosaic result sounds like the target character but with the timbral qualities of the database. Applications include voice conversion, audio mosaicing, and live performance systems.

Examples

Sven Konig’s sCrAmBlEd?HaCkZ! (2004) maps vocal beatboxing to a database of music videos. The Synful synthesizer uses a related context-sensitive sample database.

Assessment

What are the two competing cost functions in concatenative synthesis and what does each measure? Why does the quality of concatenative synthesis depend heavily on the size of the unit database?