Cache-to-Cache: Direct Semantic Communication Between LLMs (2025) (arxiv.org)

A groundbreaking approach called Cache-to-Cache (C2C) has been introduced to facilitate direct semantic communication between Large Language Models (LLMs), presenting a significant advancement in multi-LLM systems. Traditionally, these models communicate through text, which not only leads to the loss of rich semantic information but also increases latency due to token-by-token generation. C2C addresses these challenges by enabling LLMs to leverage their key-value (KV) caches for direct communication, enhancing the response quality without expanding the cache size. This method employs a neural network to merge the source model's KV-cache with that of the target model, utilizing a learnable gating mechanism to identify the most beneficial layers for communication. The implications of this innovation are substantial for the AI/ML community. C2C demonstrates an impressive 6.4-14.2% rise in average accuracy compared to individual models, and it outperforms traditional text-based communication by approximately 3.1-5.4%, all while achieving a remarkable 2.5x reduction in processing latency. This paradigm shift not only enhances the efficiency and effectiveness of LLM interactions but also opens new avenues for research and application, highlighting the potential for more sophisticated inter-model collaboration in artificial intelligence systems.