Monday, 16 November 2026 · 7 min read
Gamer trains a vision-language-action agent for long-horizon real-time game control
Gamer is a generalist agent that learns transferable keyboard-mouse control from tens of thousands of hours of gameplay, uses sparse reasoning and memory for long tasks, and applies a novel reinforcement learning method for stable policy optimization in real-time 3D games.
Based on the paper Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent by Run Luo, Kuan Zhang, Qi Deng, Yukun Chen and 23 others
Challenges of building generalist agents for complex, long-horizon games
Modern 3D games combine complex visual scenes, intricate mechanics, and long multi-stage objectives that require agents to perceive raw pixels, remember past events, plan ahead, and act through low-level keyboard and mouse inputs over hours-long interactions.
Existing game agents often rely on privileged internal states or game-specific APIs, limiting their generalization and real-time applicability. Achieving a unified agent that can handle diverse games through standard device-level interfaces, maintain memory across long tasks, and operate with low inference latency remains an open problem.
Real-time interaction adds system challenges: inference latency can cause delayed actions that degrade control quality and policy learning. Long task horizons exceed typical model context windows, complicating credit assignment and stable reinforcement learning.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three key formulas from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short