AI Cost-Performance Analysis: Open Models Deliver 75% Savings Over Top Proprietary Agents
Benchmark data shows massive cost variation for agentic knowledge work, revealing open weights models as the most cost-effective high-performance options.
Practical Summary
A new benchmark for agentic knowledge work reveals that the cost per task can vary by nearly 800 times between the most expensive and cheapest models. While the top-performing proprietary model (Claude Fable 5) is extremely costly, strong open weights alternatives like GLM-5.2 deliver near-equivalent performance for less than a quarter of the price, offering a clear pathway for cost optimization.
Why It Matters
For teams deploying AI agents for multi-step projects, model selection is a major cost driver. This data provides actionable evidence that switching to high-performing open weights models can drastically reduce API spend without a proportional drop in task quality, directly impacting operational budgets and ROI.
Understanding the Cost-Performance Landscape for Agentic AI
The AA-Briefcase benchmark evaluates AI models on complex, multi-week knowledge work projects. The results show an enormous disparity in cost per task. The leading proprietary model, Claude Fable 5, costs over $31 per task on average. In contrast, the budget-oriented DeepSeek V4 Flash costs about $0.04 per task, illustrating the extreme low end of the cost spectrum.
The most valuable finding for cost optimization is the performance of open weights models. Models like GLM-5.2 (max) and DeepSeek V4 Pro (max) are identified as having the strongest price-to-performance ratio. GLM-5.2 (max) scores only about 90 Elo points lower than the high-end proprietary model Claude Opus 4.8 (max), but does so for less than 25% of the cost.
Actionable Steps for Cost Optimization
1. **Audit Current Agent Workloads:** Identify which of your agentic AI tasks involve long-horizon, complex projects similar to the benchmark. These are prime candidates for cost review.
2. **Evaluate Open Weights Alternatives:** Use the benchmark's findings as a starting point. Test models like GLM-5.2 or DeepSeek V4 Pro on your specific tasks to measure their actual performance and cost. The goal is to find a model that meets your accuracy requirements at a significantly lower price point.
3. **Implement a Tiered Routing Strategy:** Based on your tests, route tasks of varying complexity to different models. Reserve the most expensive proprietary models only for the highest-stakes or most complex subtasks where their performance edge is critical and justified by the cost. Use the cost-effective open weights models for the majority of standard tasks.