Spring 2026 papers: LLM inference meets the memory hierarchy
AI & Data · · 5 min read
Between March and May 2026, GRC researchers and their collaborators presented four papers on one problem: long-context LLM inference is limited by how data moves through the memory hierarchy, not only by compute. The papers appeared at ASPLOS, EACL, DATE and IPDPS, and their answers range from a hardware prefetcher to KV cache pruning. A fifth paper, also at IPDPS, applies the same I/O lens to HPC workflows.