AgentsNEWHigher educationTestingHuman in loop: unclear
Preprint: an on-prem help-desk agent for Purdue HPC tickets, and where merging fell short
Researchers built an MCP assistant pairing docs RAG with a Llama-3-8B model tuned on 11,244 past ticket resolutions. On 400 held-out Purdue tickets, the fine-tuned tool led on agreement with staff answers.
Why it matters: The merged system didn't beat its tools, the authors report; returning one tool's answer scored 3.19 vs 2.66. All models stayed on-premises because ticket data can't leave.
From Drop #048 · Mon, Oct 26 · Evening