StudioI ran the same browser job 216 times with Claude Code and Codex: a script beat every agent run
Original title: I ran the same browser job 216 times with Claude Code and Codex. A script beat every agent run, and the best browser tool depended on the agent
u/maxfabiankirchner had Claude Code (Opus 5.5) and Codex (GPT 6 Astra) perform the same browser task in a business web app 216 times, with all runs passing, testing agent-browser, Playwright and Chrome DevTools over CLI+skill, CLI without skill, and MCP, 12 runs each across macOS, WSL and Windows. The best tool depended on the agent: with Codex, Chrome DevTools CLI with its skill did well on both cost and speed (median 197K input tokens, $0.53, 75s), while Claude Code showed a trade-off. The author also had Claude Code write a CDP-based Python script from an earlier session's notes, which ran at a median of 1.5s over 258 runs, with its slowest run (10.3s) still faster than the fastest agent run (59s), and suggests letting the agent work out a repeated job once, then running the script and…
216 measured runs yield median cost and time figures for each tool and connection style, plus a transferable pattern: let the agent solve a job once, then hand it to a script.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.