Windows ML is generally available (blogs.windows.com)

Microsoft has announced Windows ML is generally available as the built-in, on-device inference runtime for Windows 11, ready for production use. Designed as a hardware abstraction layer, Windows ML runs ONNX models locally across CPUs, GPUs and NPUs by leveraging ONNX Runtime (ORT) and partner-supplied Execution Providers (EPs) from AMD, Intel, NVIDIA and Qualcomm. Windows handles distribution, updates and registration of ORT and EPs, lets developers precompile models ahead-of-time (AOT), and can dynamically download the right EP for a device—reducing app overhead (often tens to hundreds of MB) and simplifying single-build deployment across diverse Windows hardware. It’s also the foundation for Windows AI Foundry and supports tooling like the AI Toolkit for VS Code and the AI Dev Gallery; it’s included in Windows App SDK 1.8.1 and requires Windows 11 24H2+. Significance: Windows ML makes local AI—lower latency, better privacy, lower cloud costs—practical at scale by unifying model format and runtime across a broad silicon ecosystem. Technical implications include fine-grained device policies (favor NPU for low power or GPU for performance), partner-optimized EPs (e.g., NVIDIA’s TensorRT EP claims >50% faster inferencing vs DirectML on RTX GPUs), and streamlined developer workflows for converting, quantizing and deploying PyTorch/ONNX models. Early adopters (Adobe, McAfee, Topaz, etc.) signal fast uptake across media, security and accessibility use cases, accelerating the shift toward hybrid cloud+edge AI on Windows devices.