Agentic systems are just query engines for unstructured data (www.daft.ai)

The author argues that modern LLMs and agentic systems are evolving from simple endpoints into declarative “query engines” for unstructured and multimodal data—think of prompts as SQL-style queries over PDFs, videos, and documents. Like SQL, which composes projections, filters, scans, joins and sorts into an optimized logical plan, LLMs compose primitives—summarization/structured extraction, scoring/evaluation, OCR, chunking/embedding, code sandboxes, tool calls and file downloads—into workflows that the model reasons about and executes (examples cited: GPT-4o, Gemini, Claude 3.5). This shifts the interface from “how” to “what,” enabling models to orchestrate downloads, parallel crawling, OCR, and sandboxed code execution rather than merely returning a forward-pass inference. The significance for AI/ML is systemic: market success increasingly depends on orchestration and runtime integration (the “compiler,” “CPU,” “memory,” and “VMs” packaged by closed models), not just raw model quality. Closed systems currently win because they bundle reasoning, tool integration and execution infrastructure; open-source efforts mostly expose the operator (forward pass) and lack the runtime, making large-scale multimodal workloads costly and brittle. The path forward is systems engineering—build richer primitives, compose them into declarative plans/workflows, and implement efficient execution engines (an “AI Spark”) so declarative pipelines can be optimized for performance and scale.