Monday, 2 November 2026 · 6 min read
R²Orch improves agent orchestration via nested self-play and skill withdrawal
R²Orch introduces a recursive self-play framework with an inner loop improving an orchestrator on self-chosen tasks and an outer loop withdrawing unhelpful skills, achieving state-of-the-art results on 12 diverse benchmarks with zero-skill deployment.
Based on the paper R^2Orch: Recursive Self-Improvement for Agent Orchestration via Self-Play within a Skill-Internalization Loop by Mingda Zhang, Yun Huang, Qiang Huang, Yanjin Li and 6 others
Challenges in learning agent orchestration and skill internalization
Agentic orchestration composes multiple agents to solve complex tasks by assigning roles and hand-offs in an agent graph, improving controllability and reusability. However, current orchestrators rely on hand-designed task curricula and in-prompt skills, which are costly and hard to scale.
Existing methods face three main challenges: (i) hand-designed curricula that do not adapt as skills are withdrawn, (ii) advantage collapse where reinforcement learning signals vanish when all rollouts yield the same outcome, and (iii) unreliable evaluation signals that judge each skill by a single noisy success score, causing skill libraries to drift.
These challenges limit orchestrators’ ability to self-improve and move toward zero-skill deployment, where the orchestrator performs well without relying on any scaffolding skills.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three key formulas from the paper, explained symbol by symbol
- Two figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short