From Lossy to Lossless Reasoning (manidoraisamy.com)

The author argues that reasoning in LLMs is effectively code generation and that improving how models turn natural language into correct code is the key to turning “lossy” probabilistic reasoning into “lossless” deterministic reasoning. Instead of linear token sequences, they propose translating user prompts into a deterministic controlled-English (ACE) representation that maps to an input graph, parsing target code into an AST output graph, and training graph transformers to convert input graphs to ASTs. Example: “place a 13' ladder against a wall at 12'” → ACE specifying ladder length and wall height → input graph → AST for math.sqrt(13*13 - 12*12). Expected benefits include big accuracy gains (FrontierMath: pure reasoning 13.5% → token code gen 26.3% → projected graph-based 35–40%), stronger bidirectional relational reasoning (e.g., parent/child queries), built-in interpretability via executable code, and 4–10x cost reductions for many tasks. Key technical points: tokenizers fragment user-defined symbols, and sequence transformers force models to reconstruct hierarchical ASTs from flat tokens; ACE gives deterministic parseability; graph transformers attend over node/edge structures and predict nodes/edges instead of next tokens. Major challenges are strict ACE parsing, lack of large ACE→code datasets, variable graph topologies, and higher computational cost from graph attention and message passing. The proposal calls for investing in ACE rewriting, aligned graph datasets, and graph-native architectures to make reasoning more accurate, verifiable, and efficient.