Monday, 19 October 2026 · 6 min read
HexaShell uses coding agents with RGB triangulation to improve robot manipulation on RoboDojo
HexaShell lets a coding agent control a dual-arm robot via a thin command interface, evolving small tools from failures using RGB camera triangulation. It achieves 39.7% success on 42 RoboDojo tasks, surpassing learned policies, but struggles with tight insertion tasks requiring precise contact.
Based on the paper HexaShell: A Thin Harness for Exploring the Limits of VLM Agents as Robot Coding Agents by Hongcheng Gao, Alex Zhou, Jay Zhu, Zelin Zheng and 12 others
Challenges in robot coding agents and precise 3D perception
Vision language models (VLMs) have recently started to compete with learned policies for robot manipulation, but controlling a physical robot requires precise 3D understanding and motion execution, which is difficult from RGB images alone.
Coding agents traditionally fix failing software tests by reading errors and editing code; applying this to robots means the agent must interpret visual failures and write or fix small programs (tools) that control robot actions.
A key difficulty is converting 2D pixels into accurate 3D positions for manipulation, as monocular depth estimation is often inaccurate, and precise contact tasks require centimeter-level accuracy.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Two figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short