Exponential Moving Average of weights produces smoother RAVE models by averaging checkpoints over time

The EMA callback maintains a shadow copy of all model parameters as an exponential moving average: ema_weights = factor * ema_weights + (1-factor) * current_weights. During validation, the live weights are swapped for EMA weights so the validation metrics and saved audio samples reflect the smoother, averaged model. At export time, --ema_weights loads the EMA parameter set rather than the last training checkpoint. EMA models typically sound more stable and artifact-free than raw checkpoints because short-term weight fluctuations (from adversarial instability) are smoothed out. A factor near 0.999 means the EMA weights trail the live weights by roughly 1000 steps.

Examples

rave train --config v2 --ema 0.999 .... Export: rave export --run /path --ema_weights --streaming.

Assessment

If EMA factor = 0.999 and training has 1,000,000 steps, approximately how many steps of ‘memory’ does the EMA represent? Why prefer EMA weights at export over the final checkpoint?