Friday, 16 October 2026 · 6 min read
OpenGCUA enables models to flexibly use GUI, CLI, and code execution for computer tasks
OpenGCUA defines and implements a generalist computer-use agent that decides what to observe, how to act, and how to organize work across GUI, accessibility, CLI, and API interfaces, improving task scores by 19.6 points and halving steps on OSWorld-v2.
Based on the paper OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use by Hanzhang Zhou, Long Li, Quyu Kong, Lei Wang and 11 others
Expanding from GUI-only agents to generalist computer-use agents
Computer-use agents traditionally operate by observing screenshots and performing mouse and keyboard actions, limiting them to GUI-only interaction.
Recent advances combine GUI interaction with programmatic interfaces like shell commands and APIs, enabling more efficient task completion.
Generalist computer-use agents (GCUAs) are defined as systems where the model decides how to observe, act, and organize work across visual and programmatic interfaces, but prior implementations like GPT-6 Astra remain closed, limiting study.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- The key formula from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short