Skip to content
Free since 2015. Built for the engineers who keep things running.

cat /articles/your-ai-coding-agent-could-cost-5x-more-depending-on-the-harness-benchmark-finds.md

Platform Engineering

Your AI coding agent could cost 5x more depending on the harness, benchmark finds

The same open-weight model can cost up to five times more to run depending on which coding agent harness it is paired with, according to a 22-configuration benchmark. Most inference spending went to failed or timed-out runs.

The New Stack17 Nov 2026 · 4 min read · summary

article --summary

The summary

A new benchmark tested 22 coding agent harness setups across five open-weight models, running them many thousands of times on terminal-based tasks. The authors found that the model itself explains most of the cost differences, but the harness still matters a great deal, and a setup that is cheapest for one model can be among the most expensive for another. The study also looked at where the money goes. Roughly two-thirds of estimated spending landed on runs that failed verification or timed out, and the number of agent steps was the only factor that clearly tracked cost per successful task. The simpler harnesses performed well, but the study did not test their design choices separately, so it cannot confirm why they were cheaper.

Editorial note: This is a summary of an article first published by The New Stack. The full piece is on their site.

Reading now

Topic
Platform Engineering
Source
The New Stack
Published
17 Nov 2026
Format
Curated summary

Have a take on this?

Write it up and submit it. It goes on the list once an editor approves it.

Submit an article

articles --latest

Latest articles

New from the blogs we follow and the community.

All articles

newsletter --weekly

DevOps Weekly, minus the noise.

The best reads, new jobs and upcoming events, in one email a week.