LLMs and Beyond: All Roads Lead to Latent Space (aiprospects.substack.com)

An influential essay updated June 28, 2025, argues that “latent space” — not tokens or crude statistical pattern‑matching — is the central concept for understanding LLMs and modern AI. The author contends that token prediction is merely a training objective; the real work happens in high‑dimensional latent or semantic spaces where Transformers encode meaning as geometric patterns: concepts as directions, categories as clusters, and reasoning as transformations of activation vectors across layers. Empirical analyses (e.g., sparse autoencoders) show per‑position embeddings decomposing into dozens of weighted “concept vectors” drawn from vector‑vocabularies of millions; modern models operate in thousands of dimensions (4096 dims cited for Llama), giving latent space capacity that far exceeds biological synapses. Technically and practically, this framing explains why deep learning generalizes across domains: AlphaFold and image or robot models all learn domain‑specific latent spaces and mappings, and multimodal alignment (e.g., CLIP‑style text↔image embeddings) enables cross‑domain reasoning. The essay advocates moving beyond text‑chunk RAG to Large Knowledge Models (LKMs) that store persistent, updatable knowledge as latent‑space bundles retrievable by Transformers. LKMs promise explicit provenance, incremental updates, smaller reasoning models, and cumulative, auditable knowledge — changes that could unlock nonlinear capability gains as representation quality and multimodal integration improve.