Daily papers

Thursday, 19 November 2026 · 7 min read

Agent evolution often fails due to noisy evaluation, not poor proposals

Agent evolution methods generate candidate improvements but struggle to reliably identify better artifacts due to noisy evaluation, causing evolution to stall and rankings to be unstable. Large-scale tests show selection is near chance at picking improvements, calling for better evaluation resolution.

Based on the paper The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements by Yuanhao Zeng, Shaocheng Lan, Ao Lu, Kuang Yan and 7 others

Challenges in agent evolution and evaluation noise

Agent evolution improves large language model (LLM) agents by iteratively proposing candidate artifacts—such as prompts, skills, memories, or agent harnesses—and selecting those that score higher on task benchmarks. This propose-evaluate-select loop has shown promise but also exhibits instability: methods trade wins across papers, and evolution often stalls on certain tasks.

A key difficulty is that evaluation scores are noisy due to variability in agent behavior and task execution. This noise makes it hard to reliably identify true improvements among candidate artifacts. Consequently, selection may accept harmful or neutral candidates by mistake, causing evolution to fail to accumulate real gains.

The paper formulates the Resolution Hypothesis: evolution stalls not because proposers fail to generate better artifacts, but because noisy evaluation lacks the resolution to distinguish small true improvements from noise. This hypothesis explains why reported method rankings are unstable and why evolution often fails to bootstrap on challenging benchmarks.

Sign in to read the whole paper

You have read the opening. The rest is free with a Bezen account, and no card is needed.

In the rest of this paper

  • How the method works, step by step, with a flow chart
  • Two key formulas from the paper, explained symbol by symbol
  • Three figures from the paper, and how to read them
  • How it was tested, what was found, and where it falls short
Agent evolution often fails due to noisy evaluation, not poor proposals · Bezen