Author name: Ranit Karmakar, Jayita Chatterjee

arXiv:2605.02038v1 Announce Type: new
Abstract: Single-prompt accuracy is the dominant way to benchmark language models, but it can miss reliability failures that matter. We evaluate a 15-model open-weight corpus, with the main reliability analyses fo…

arXiv:2605.00334v1 Announce Type: new
Abstract: Production agentic systems make many model calls per user request, and most of those calls are short, structured, and routine. This raises a practical routing question that existing evaluations do not di…