Agents in AI infrastructure

Preprint: agents that drop their own KV cache cut cost 27.7% vs summarizing

Researchers propose cache engineering: agents drop or reposition cached context instead of summarizing history. A rolling-drop policy raised average benchmark scores 1.7 points and cut cost 27.7%, the abstract says.

Why it matters: Long agent runs get expensive as context grows. The authors pair the method with an AKV serving engine, suggesting the serving layer, not just prompts, can help.