Are Harnesses Worth the Cost?
martinfowler.com
A coding-agent harness that added contract guides and executable feedback increased token use, time, and cost while the agent still met the same acceptance criteria in repeated trials. With GPT-6 Sol, the harness used 10.5 times as many tokens and took nearly 3 times as long, yet both harnessed and unassisted runs succeeded in all 10 runs. The result was no observable improvement in final application reliability, so the harness failed to justify its added cost in this setup.
Why it matters
The article begins by evaluating harnesses in the context of running tests on a Java API service. It frames the discussion around whether the extra cost is justified in ten runs of the service.