Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents (github.com)

Magnitude, a new open-source inference engine developed by Y Combinator’s Summer 2025 batch, has launched with a focus on self-optimization for specific hardware. This engine allows users to run open models significantly faster—up to 2x quicker than llama.cpp—by tuning its kernels directly on the user's device. Designed for compatibility with a variety of hardware configurations, including Apple Silicon, NVIDIA, AMD, and even CPUs, Magnitude offers a hassle-free connection to popular agents like Pi, OpenCode, and Hermes with just one click. The significance of Magnitude lies in its ability to maximize hardware efficiency, featuring optimizations that yield remarkable performance improvements, such as 92% faster decoding on Metal and 19% on CUDA. The engine manages memory usage effectively, reducing the footprint by 27% per agent, and allows for rapid concurrent sessions without slowdown. As an open-source project governed by the Apache 2.0 license, Magnitude emphasizes privacy and cost-effectiveness, as users retain full control of their data, ensuring a seamless experience without internet connectivity once models are downloaded. This innovation could significantly impact the AI/ML community by enhancing the accessibility and efficiency of running complex models on a wide range of devices.