Limitations of the noise model of roundoff for the FFT

Abstract

The general assumptions made about roundoff noise are that its samples form a signal-independent white sequence, and they are uniformly distributed between \pm q/2, where q equals the least significant bit (LSB). While these are often true, strange cases may appear, e.g. misleading peaks can occur in the spectrum. In this paper the roundoff error of the fixed-point and floating point fast Fourier transform is investigated. It reproduces the results of Welch (1969) with modern tools, revisits his simulations, and investigates the consequences of the violation of the above assumptions for almost pure sine waves. The maximum amplitude of spurious peaks is determined and the amount of the decrease in the dynamic range is given.