LangChain: Harness-Only Changes Lift Deep Agents Coding Agent from Top 30 to Top 5 on Terminal Bench 2.0
Summary
- • LangChain's deepagents-cli rose from Top 30 to Top 5 on Terminal Bench 2.0
- • Score improved from 52.8 to 66.5 (+13.7 points) with the model held fixed
- • Only the harness changed: system prompt, tools, and middleware around gpt-5.2-codex
- • Results are LangChain's own benchmark claim, not independently verified
Details
deepagents-cli moved from Top 30 to Top 5 on Terminal Bench 2.0
The benchmark has 89 tasks. The jump came without any change to the underlying model.
Score rose from 52.8 to 66.5, a gain of 13.7 points
LangChain attributes the gain entirely to harness changes, with gpt-5.2-codex fixed.
Tuned three harness knobs: system prompt, tools, and middleware
Middleware means hooks around model and tool calls that alter behavior without changing the model.
Experiments used Harbor for orchestration, Daytona sandboxes, and LangSmith traces
Traces captured actions, latency, token counts and costs for failure analysis.
LangChain built a repeatable Trace Analyzer agent skill for harness improvement
It fetches traces, spawns parallel error-analysis agents, synthesizes findings and drives targeted changes; to be published soon.
Common failure: agent wrote a solution, re-read its code, and stopped without testing
The fix was prompt guidance for planning, building with verification, and a self-verify loop.
Stat = results, Tech Info = detail, Infrastructure = tooling, New Tech = new capability, Research = findings
What This Means
LangChain's report suggests how an agent is wrapped can matter as much as the model, since a 13.7-point gain came with the model unchanged. A trace-driven loop that fixes repeated failures in the harness is a repeatable approach. The numbers are the vendor's own claim.