Friday, 23 October 2026 · 7 min read
OpenDS-World benchmark evaluates AI deep search on real Web tasks with 39.0% top accuracy
OpenDS-World introduces 200 complex real-world Web search tasks requiring multi-step evidence gathering across diverse modalities and formats; the best AI model achieves 39.0% accuracy, while humans reach 21.1%, highlighting persistent challenges in autonomous Web deep search.
Based on the paper Deep Search in Real-World Web Environments by Xiaomin Yu, Hongliang Chen, Zhe Huang, Shilin Yan and 21 others
Challenges of autonomous deep search in complex, heterogeneous Web environments
Deep Search agents aim to autonomously acquire, analyze, and integrate information to solve complex research tasks by searching the Web. However, real-world Web environments are complex because information is spread across diverse sources, modalities (text, images, audio, video), and heterogeneous formats (HTML, PDF, MP4, Parquet), often requiring interactive access like navigation or parameter input.
Existing benchmarks and evaluations do not fully capture these complexities, limiting systematic assessment of agents’ capabilities in multi-step evidence acquisition, cross-modal reasoning, and interactive Web navigation. This gap motivates the need for a new task and benchmark that reflect real-world Web search challenges.
The OpenDS task and OpenDS-World benchmark address this by requiring agents to autonomously explore the live Web, discover and connect evidence from multiple independent sources, and produce unique factual answers, emphasizing continuous evidence acquisition and integration across modalities, formats, and interactive access mechanisms.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short