A Mathematical Framework for Scalable, Parallel, High-Dynamic-Range Computation (arxiv.org)

Researchers introduce generalized orders of magnitude (GOOMs), a mathematical representation that extends traditional orders-of-magnitude reasoning and subsumes IEEE floating-point as a special case to enable stable compounding of real numbers across vastly larger dynamic ranges. Paired with an efficient custom parallel prefix scan designed for GPUs, GOOMs let inherently sequential accumulation operations be executed natively and in parallel without catastrophic underflow or overflow. The authors provide an implementation and demos showing that tasks previously considered impractical or impossible with standard floating point now become feasible and performant. Key technical contributions and demonstrations include: (1) reliably compounding long chains of real matrix products far beyond floating-point limits; (2) computing spectra of Lyapunov exponents in parallel with orders-of-magnitude speedups, using a novel selective-resetting trick to prevent state colinearity; and (3) capturing long-range dependencies in recurrent neural networks with full (non-diagonal) recurrent states computed via parallel prefix scans, removing the need for ad hoc stabilization. Collectively, GOOMs plus parallel scanning offer a scalable, numerically robust alternative to conventional floating point for high-dynamic-range ML, dynamical-systems analysis, and financial or scientific workloads that require safe, parallelizable accumulation over long sequences.