nn~'s circular buffer amortizes neural model compute across time, at the cost of added latency

Neural models are computationally intensive and cannot run at sample-level granularity. nn~ uses a circular buffer: audio accumulates until a buffer of the configured size is filled, then the model processes the whole buffer in one pass. A larger buffer size reduces CPU load (fewer model calls per second) but adds latency equal to the buffer duration. The buffer size is set as an integer argument after the method name (e.g. nn~ model.ts forward 4096). On Windows, the circular buffer is disabled by default due to a PyTorch threading memory-leak bug, resulting in higher CPU load. Setting buffer size to 0 disables the mechanism entirely.

Examples

nn~ rave.ts forward 2048 — 2048-sample buffer at 44100 Hz ≈ 46 ms latency. nn~ rave.ts forward 8192 — 8192 samples ≈ 186 ms but lower CPU. For interactive live use, prefer the smallest buffer that keeps CPU below 80%.

Assessment

Given a model running at 44100 Hz with a buffer of 4096 samples, calculate the added latency in milliseconds. Explain the CPU-vs-latency tradeoff and what to do on Windows when the circular buffer is disabled.