LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs: LLM serving is increasingly accelerated by...
Position-independent caching (PIC). Existing PIC methods, however, are built for full-attention models, where a token-indexed KV cache underlies its core operations: matching reusable token chunks, concatenating their KV entries, and selectively recomputing a few tokens to restore cross-chunk context. Hybrid LLMs...
来源简报
LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs: LLM serving is increasingly accelerated by position-independent caching (PIC). Existing PIC methods, however, are built for full-attention models, where a token-indexed KV cache underlies its core operations: matching reusable token chunks, concatenating their KV entries, and selectively recomputing a few tokens to restore cross-chunk context. Hybrid LLMs...