The Shift from ML Engineering to AI Engineering (bryananthonio.com)

Chip Huyen’s AI Engineering chapter 1 reframes the practitioner’s role: instead of classical ML engineering (curating data and training models from scratch), AI engineering centers on adapting large pre-trained “foundation” models—LLMs and multimodal models—to specific applications. That lowers the barrier to entry because teams don’t need to build models end-to-end, but it also introduces new operational and evaluation challenges. The chapter clarifies how modern language models use tokenization and distinguishes masked language models (fill-in-the-blank) from autoregressive models (next-token prediction), the latter powering most generative AI features today. Practically, integrating foundation models into products follows three main paths—prompt engineering, retrieval-augmented generation (RAG), and fine-tuning—each with trade-offs in cost, complexity, and performance. Evaluating model behavior for specific, often open-ended tasks is hard, demanding specialized “evals.” Running models at scale raises inference cost and latency concerns, motivating techniques like distillation, quantization, and parallelism to optimize performance. The upshot: AI engineering shifts the skillset from bespoke model training to prompt design, retrieval systems, model selection, evaluation frameworks, and robust inference infrastructure—making these capabilities the new levers for building reliable, cost-effective AI applications.