Multi-scale SpectralLoss compares audio at multiple FFT sizes to balance time and frequency resolution

DDSP’s SpectralLoss computes the reconstruction error between target and synthesized audio by comparing spectrograms at multiple FFT sizes (e.g., 2048, 1024, 512, 256, 128, 64). Each scale captures different time-frequency tradeoffs: large FFTs resolve harmonics but smear transients; small FFTs resolve transients but blur harmonics. The loss sums weighted combinations of linear magnitude, log-magnitude, time-difference, frequency-difference, and cumulative-frequency (Wasserstein-style) terms. Log-magnitude loss is more sensitive to quiet components; linear magnitude is more sensitive to peaks. Using multiple scales avoids the optimization landscape where the signal is correct at one scale but wrong at another.

Examples

spectral_loss = ddsp.losses.SpectralLoss(
    fft_sizes=(2048, 1024, 512, 256, 128, 64),
    mag_weight=1.0,
    logmag_weight=1.0)
loss = spectral_loss(target_audio, synthesized_audio)

Assessment

Why does comparing audio at only one FFT size (e.g., 2048) risk failing to reconstruct transients? What does logmag_weight emphasize that mag_weight does not?