home/ atoms/ quantization-error-basics

Quantization error is the rounding difference between the true signal amplitude and the nearest sample value

Digital audio stores each sample as an integer at one of 2^n levels (where n is bit depth). Any signal amplitude that does not fall exactly on a quantization step is rounded to the nearest representable value, introducing an error of up to ±½ LSB (least significant bit). At high bit depths (24-bit) this error is well below the noise floor of any practical analog circuit. At low bit depths, quantization error is audible as distortion correlated to the signal. The error is bounded (max ±½ LSB) and deterministic for a given sample value, which is what makes dither effective: adding a small noise signal before quantization decorrelates the error from the signal.

Examples

A 16-bit converter has 65,536 levels; a full-scale 1 kHz sine has roughly ±32,768 steps with quantization error near ±0.5 steps, corresponding to about -96 dBFS. A 4-bit converter has only 16 levels, producing very audible stepping.

Assessment

For a 16-bit audio file, calculate the maximum quantization error in LSBs. Then explain what happens to that error when dither is applied before quantization.

“all of the error is confined to the ± 1 lsb area. For a signal at or near full scale and 24-bit, the total error is well below the noise level of any analog audio circuit playing it back.”
corpus · earlevel-engineering-digital-audio-dsp-blog · chunk 3