Information theory for complex systems scientists: What, why, and how (www.sciencedirect.com)

A new primer titled "Information theory for complex systems scientists: What, why, and how" presents a practical guide to applying information‑theoretic tools to study dynamics, interactions, and emergence in complex systems. It frames information theory—notably entropy, mutual information, conditional/multivariate information, transfer entropy (directed information flow), and the information bottleneck—as core concepts for quantifying uncertainty, dependencies, and directed influence without strong model assumptions. The piece emphasizes why these measures matter for AI/ML: they provide model‑agnostic diagnostics for feature relevance, causal discovery, interpretability, network reconstruction, and for probing learning dynamics in trained models and multi‑agent systems. Technically, the primer surveys common estimators and pitfalls—discretization and binning, k‑nearest‑neighbor (Kraskov) and kernel estimators for continuous data, Bayesian approaches, bias correction, surrogate testing, and the importance of sample size and stationarity. It also highlights advanced topics like partial information decomposition (PID) to separate redundant, unique, and synergistic information, transfer entropy for directed interaction inference, and multi‑scale or integrated information measures for emergent behavior. Practical implications include guidance on estimator choice, validation with surrogates, and combining information measures with causal and mechanistic modeling to improve interpretability, robust feature selection, and the study of collective intelligence in AI systems.