In plain words: Instead of writing out words to each other, two AI models share their internal memory directly, with a small network blending one model's stored context into the other's. This beat text-based exchange on accuracy and ran about 2.5 times faster.
Abstract · Cache-to-Cache: Direct Semantic Communication Between Large Language Models
Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at https://github.com/thu-nics/C2C.
Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, Yu Wang
arXiv:2510.03215 · cs.CL, cs.LG · submitted Oct 3, 2025 · updated Mar 2, 2026
abstract · pdf · html · Published in ICLR'26
If multiple models can use cache representations for this kind of enrichment, the KV cache representations of different models must be somewhat compatible.
What stops us then from going a step further, and producing a model family where all models are "KV aligned", and each model can utilize the KV cache of other models directly?
So, an "expensive" reasoning model can use its full faculties to plan, but "delegate" simple subgoals to a smaller model. That smaller model can access the large model's intent directly, as rich KV cache representations - with no prefill recompute and no associated "handover" latency. Or, likewise, a "cheap" small model can generate a diminished but highly compact KV cache that the "expensive" model can then operate on - for example, for skimming a large file for shallow patterns.