The Case for Compact AI – Communications of the ACM (dl.acm.org)

Tim Menzies argues in Communications of the ACM that the AI community — especially software engineering (SE) researchers — should stop assuming large language models (LLMs) are the default or best path. He highlights that only ~5% of SE LLM papers examined alternatives, and points to “funneling” in software (complex internals yielding a few dominant outcomes) which lets simple models perform well. Using the MOOT benchmark (63 diverse SE multi-objective tasks drawn from datasets with hundreds of thousands of examples and up to 1,000 settings), Menzies shows active learning can be fast, cheap, and explainable while achieving near-optimal results. He presents BareLogic, a pedagogical Bayesian active learner: label N=4 random examples, rank them by “distance to heaven” (ideal targets), split into best vs rest, train a two-class Bayes classifier, pick the unlabeled X that maximizes log P(best|X) − log P(rest|X), label X, increment N and repeat until stopping. Across tasks, 8 labels yielded ~62% of optimal, 16 ≈80%, 32 ≈90% (64 gave little extra gain). The experiments ran in minutes on a laptop. Menzies argues this approach tackles LLM concerns—training time, energy, specialized hardware, testability and explainability—by producing small, auditable models (regression trees) and supporting human-in-the-loop decisions, making a compelling case for “compact AI” instead of defaulting to planetary-scale models.