2026-09-20T04:30:01Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/129176 2025-10-20 col_2142_5131 col_2142_10761 com_2142_5130 com_2142_10755 com_2142_234

application/pdf en text Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms The student, Akshat Sharma, accepted the attached license on 2025-04-18 at 14:17. The student, Akshat Sharma, submitted this Thesis for approval on 2025-04-18 at 14:24. This Thesis was approved for publication on 2025-04-20 at 16:29. DSpace SAF Submission Ingestion Package generated from Vireo submission #21668 on 2025-10-19 at 18:09:09 2025-05 Thesis https://hdl.handle.net/2142/129176 Copyright 2025 Akshat Sharma Pushing the limits of long context LLM inference via KV cache compression Sharma, Akshat 2025-04-20 Zhang, Minjia Systems for LLM LLM Inference eng Efficiently deploying/serving LLMs has become remarkably challenging due to their excessive memory and computational requirements. A critical bottleneck in LLM inference is the memory footprint of the Key-Value (KV) cache, particularly in tasks involving long-context understanding and generation. To address these challenges we introduce MiniKV, a hybrid KV cache optimization technique which compresses the KV cache by combining token eviction and 2-bit quantization. Our approach aims to significantly reduce memory usage while maintaining high accuracy on downstream tasks such as question answering, summarization, code generation, and retrieval. Our evaluations demonstrate that MiniKV achieves an 86% reduction in KV cache size while recovering over 98.5% accuracy across downstream tasks. This sets a new state-of-the-art in balancing accuracy and compression, with notable improvements in inference latency and throughput. Siebel School Comp & Data Sci Computer Science University of Illinois Urbana-Champaign M.S. Thesis