TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration: Long-context prefill in large language models...
(LLMs) incurs substantial computation and memory traffic because dense self-attention computes quadratic query-key scores. Existing methods either use a uniform low-precision path or select token interactions, leaving spatial precision routing over hardware-aligned score tiles outside fused dense attention. We...
Source Brief
TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration: Long-context prefill in large language models (LLMs) incurs substantial computation and memory traffic because dense self-attention computes quadratic query-key scores. Existing methods either use a uniform low-precision path or select token interactions, leaving spatial precision routing over hardware-aligned score tiles outside fused dense attention. We...