Cursor and Together AI deliver real-time, low-latency inference at scale (www.together.ai)

Cursor, an AI-driven coding platform, has partnered with Together AI to deliver real-time, low-latency inference at scale, leveraging NVIDIA's Blackwell architecture. This collaboration addresses the critical need for in-editor agents that respond instantaneously as developers code, maintaining contextual accuracy for suggestions and refactoring. To achieve this, they optimized the entire inference stack to meet stringent latency requirements, enabling a seamless developer experience. Key advancements include the deployment of the GB200 NVL72, which supports higher memory bandwidth and tensor throughput, as well as custom Tensor Core instructions for optimized performance. The partnership significantly reduces the time from developing AI model weights to testing in production through an efficient quantization pipeline, using NVIDIA's TensorRT. This method ensures that while weights are compressed for fast inference, the quality of suggestions remains high, which is crucial in coding environments. With their current focus on further enhancing throughput and utilization on the NVIDIA platform, Cursor and Together AI are set to scale their services, maintaining an agile development cycle that promises continuous improvement and efficiency in AI-assisted coding tasks.