Detecting Beats, Onsets and Pitch for Event Triggers

Continuous loudness and spectrum curves are fine for wobbling a blob, but a live audiovisual set lives or dies on discrete events: the flash that lands exactly on the kick, the scene cut on a drop, the glyph that spawns when the vocalist enters. This module turns the raw features from the prerequisite module into a trustworthy event stream — the difference between visuals that feel synced and visuals that feel coincidental, whether you are VJing in a club with a noisy DI feed or scoring a browser piece against a known track.

The arc starts supported: wire a naive detector using beat detection via amplitude threshold against a steady four-on-the-floor loop, and watch it fail on sustained passages and quiet intros. That failure motivates the decaying-cutoff-plus-hold-window pattern from robust beat detection, which you tune (floor, hold frames, decay rate) until double-triggers vanish. From there you generalize: onset detection by energy and spectral flux catches events a bass-band threshold misses, and autocorrelation gives you a pitch readout by finding the lag where the signal resembles itself. The capstone assembles all three into one detector and then confronts it with ground truth — pre-rendered, timestamped analysis of the same track — so you can measure where your live detector drifts, mis-fires, or lags.

Every required atom gates that build: the two threshold procedures produce the clean beat trigger, onset detection and autocorrelation produce the flag and pitch outputs, and pre-rendered analysis supplies the offline comparison leg. As supporting context, the MIR overview frames your hand-rolled features as the front end of a much larger machine-listening pipeline — useful perspective when judging what live vs offline accuracy actually means. Drill the threshold-tuning loop until it is reflexive; you will redo it for every new track and room.