AV coherence and sync strategies: making sound and image agree

This module addresses the question that comes immediately after “how do I wire a band to a parameter?” — namely, why does the patch still feel disconnected? Reactive motion is necessary but not sufficient for AV coherence: a patch can be fully wired and still read as noise because energy, spectral, and section axes are misaligned. The whole task here is diagnosing and fixing a real patch, then adding two positive sync strategies that work within the rig’s constraints.

The arc opens with the three-axes framework. Coherent AV means the image and the sound agree simultaneously on energy (louder music → more visual activity), spectral balance (bass drives large/slow elements, highs drive fine/fast detail), and section (breakdown → sparse visuals, drop → dense visuals). A patch can pass spectral but fail energy — for example, an inverted coupling where louder music calms the image — or pass both reactive axes but fail section because the Hydra sketch stays equally dense through intro and climax. The three-axes vocabulary gives the learner a concrete checklist rather than “something feels off”.

Section agreement is the most common failure, and its fix is unintuitive: it cannot be automated by the FFT. The 4-bin bridge carries instantaneous band energy but no section marker, drop signal, or arrangement position. The only way to achieve section-level coherence is for the performer to edit both sides together — simplify the Hydra sketch at the same moment the Strudel arrangement drops to a breakdown. The capstone makes this explicit: the section-level edit is a named step, not a side-effect.

The clock section covers the fundamental constraint. Strudel’s cycle transport and Hydra’s time/bpm are completely independent; the only signal crossing the bridge is band energy. Setting Hydra’s bpm to match the music tempo produces a frequency match but the phases drift — after five minutes the visual step lands a full beat late. The blocked-class atom enumerates the whole family of couplings that share this root cause: onset flash, kick-specific sidechain, beat-locked scroll, downbeat scene cut — all require event-level signal that the display-rate FFT cannot supply. Naming the block class once prevents repeated failed attempts at these ideas.

Because clock lock is unavailable, the positive strategies matter. Energy-driven visual motion (driving oscillation rate from band energy rather than from bpm) produces a tighter in-time feel because the energy rises and falls with the groove; the learner contrast-tests this against bpm-matched motion in the first exercise. The shared-LFO-rate lockstep is the most reliable alternative sync: a free-running audio tremolo and a Hydra brightness-pulse set to the same rate value move in lockstep without any clock connection — rate equality is enough when both sides run at the same frequency. The energy-to-motion-rate arc provides section-aware continuous sync: as the arrangement builds energy, visual motion rate rises, so both domains accelerate together through a build.

The feedback-loop/feedback-trail analogy is the cross-domain structural insight at the center of the module. The music-side feedback loop (a signal fed back into itself with sub-unity gain, producing echo tails or flanger thickening) and the visual feedback trail (a frame fed back into the render with opacity < 1, producing smear or motion blur) are the same integrate-with-decay structure. Recognizing this lets the learner compose with structural echoes: a dub delay in the audio reads coherently alongside a feedback trail in the visuals because both implement self-feedback. It also provides a model for other cross-domain structural analogies.

The two supporting atoms enrich the analogical register but do not gate the capstone. The ADSR/physics-simulation analogy (both integrate an impulse over time) and the voice-concurrency/particle-system analogy (many concurrent self-timed agents in both domains) are documented for learners who want to go deeper into structural AV composition; neither is needed to diagnose, fix, and augment the patch in the capstone.