Llama.cpp v1.0 on an RTX 3090: up to 55% faster with speculative decoding (homebrewserver.club)

Llama.cpp has officially launched version 1.0 on November 13, 2026, marking a significant milestone for the tool that allows users to run large language models on personal devices. This update, which improved performance on NVIDIA's RTX 3090 GPU, introduces several enhancements, including the removal of outdated functions, new model options, and an FP8 setting for the context cache. Notably, speculative decoding has been a game-changer, allowing the 27B model to achieve speeds that are 52% to 55% faster than the previous pre-v1 build, delivering exceptional output capabilities for parallel request handling. The implications of this update are substantial for the AI/ML community, particularly for developers utilizing llama.cpp in their applications. With the new CUDA kernel, users can process multiple requests simultaneously, resulting in up to 22% more output compared to previous versions. The delineation of versioning now communicates potential breaking changes more clearly, streamlining future updates and maintenance. Overall, version 1.0 not only enhances processing speed but also reinforces llama.cpp's position as a key tool for deploying AI models efficiently in personal and server environments, encouraging broader adoption among developers and researchers in the field.