Imp, an LLM Server for RTX 50 GPUs, Written by Claude Code (github.com)

A new LLM server called "imp" has been launched, specifically designed for NVIDIA's Blackwell architecture, which includes RTX 5050 to 5090 GPUs. Built by Claude Code, this server operates on individual Blackwell cards and allows users to run large language models efficiently without needing an extensive installation process—just a single Docker command. The server supports direct integration with popular APIs from OpenAI and Anthropic and comes with built-in chat functionalities suitable for coding applications. With the ability to utilize native FP4 math and tensor cores, imp is optimized for speed and performance, achieving notable metrics such as 1979 tokens per second for 32 simultaneous users. This development is significant for the AI/ML community as it democratizes access to high-performance language models for users with single GPU setups, enabling them to leverage advanced models without the complexities typically associated with deployment. By maintaining a focus on a specific GPU generation, imp promises optimized performance and resource utilization, with features like shared memory caching and continuous batching that allow for higher efficiency. As more developers seek to harness the power of AI without hefty infrastructure investments, solutions like imp may accelerate the adoption and experimentation of machine learning applications.