Epiplexity: Rethinking Information for Computationally Bounded Intelligence (arxiv.org)

A groundbreaking study introduces the concept of "epiplexity," rethinking how information is understood and utilized by computationally bounded intelligence. Traditional metrics, like Shannon information and Kolmogorov complexity, have shortcomings, as they assume unlimited computational capacity and fail to address the actionable content from data. This research outlines three paradoxes in information theory, emphasizing that while deterministic transformations don't increase information, epiplexity allows researchers to better evaluate the structural content of data, excluding unpredictable elements. Significantly, this new framework not only illustrates that information can be generated through computation but also emphasizes the importance of data ordering in likelihood modeling. Practically, the paper proposes methods for estimating epiplexity, which can differentiate data sources and improve out-of-distribution generalization in machine learning tasks. By providing a theoretical basis for data selection rather than model selection, epiplexity offers a fresh perspective for optimizing learning systems—highlighting how to select, transform, or generate data to enhance performance effectively. This innovation has the potential to reshape strategies in AI/ML, enabling researchers to extract more value from existing datasets.