Bezen

Your daily digest

Research papers, made simple to understand

The AI and machine learning papers people are reading most, each one rewritten in plain words: the problem, the idea, what was found and why it matters.

Benchmarks & Evaluation9 papers, newest first

    • AI Agents
    • Benchmarks & Evaluation

    OpenDS-World benchmark evaluates AI deep search on real Web tasks with 39.0% top accuracy

    OpenDS-World introduces 200 complex real-world Web search tasks requiring multi-step evidence gathering across diverse modalities and formats; the best AI model achieves 39.0% accuracy, while humans reach 21.1%, highlighting persistent challenges in autonomous Web deep search.

    Paper: Deep Search in Real-World Web Environments · Xiaomin Yu, Hongliang Chen, Zhe Huang and 22 others

    152 upvotes on Hugging Face23 Oct 20267 min readRead the article

    • Image & Video Generation
    • Benchmarks & Evaluation

    StreamEval benchmark measures interactive streaming video generation across five tasks

    StreamEval introduces a unified benchmark with 300 scripted test cases and ten metrics to evaluate real-time interactive video generation models on avatar dialogue, avatar performance, world generation, instruction editing, and reference editing, revealing challenges in request execution and state consistency.

    Paper: StreamEval: Evaluating Interactive Streaming Video Generation Models · StreamEval Team

    363 upvotes on Hugging Face22 Oct 20267 min readRead the article

    • AI Agents
    • Benchmarks & Evaluation

    LLM agents approach but do not yet match expert-level operations research solutions

    OR-License benchmark tests 11 large language models on 31 real-world OR tasks with client interaction; best model achieves 88.39% feasibility and 0.87 deployment-adjusted quality versus expert reference of 1, revealing gaps in sustained information acquisition and problem formulation.

    Paper: OR-License: Evaluating LLM Agents as Real-World Operations Research Practitioners · Peisong Wang, Feixue Liu, Yize Meng and 16 others

    196 upvotes on Hugging Face20 Oct 20266 min readRead the article

    • AI Agents
    • Benchmarks & Evaluation

    BizBench evaluates long-horizon agents on valid, constraint-compliant business task execution

    BizBench is a benchmark of 50 real-world business tasks with 789 documents and 601 tools, testing agents on both final success and intermediate compliance; top models like Claude Opus 5 reach only 64.0% strict success, revealing gaps in execution validity and rule adherence.

    Paper: BizBench: Unveiling Hidden Challenges in Deploying Long-Horizon Agents for Real-World Business · Enhan Zhao, Yongqi Tong, Xuejun Yu and 6 others

    169 upvotes on Hugging Face19 Oct 20267 min readRead the article

    • Multimodal & Vision
    • Benchmarks & Evaluation

    OmniCompass benchmark evaluates multimodal video grounding and understanding comprehensively

    OmniCompass introduces paired text and text+image target specifications across vision-language and audio-visual tracks, assessing temporal grounding, spatial grounding, and question answering on 820 videos, revealing gaps in grounding, audio-visual integration, and cross-specification consistency.

    Paper: OmniCompass: A Holistic Benchmark for Omni-Modal Video Grounding and Understanding · Xuyang Liu, Shihao Wang, Xinyu Wei and 5 others

    567 upvotes on Hugging Face16 Oct 20266 min readRead the article

    • AI Agents
    • Benchmarks & Evaluation

    AgentSWE benchmark reveals coding agents often fail to build agents meeting user requirements

    AgentSWE evaluates coding agents on creating, editing, and optimizing software agents from natural-language requirements, finding they frequently overfit, produce incomplete runtimes, and overclaim success, with top builders scoring under 31.6 on held-out tests.

    Paper: AgentSWE: Can Coding Agents Build the Agent You Actually Want? · Jiahao Wang, Hongjin Qian, Yuyang Hu and 2 others

    202 upvotes on Hugging Face13 Oct 20266 min readRead the article

    • Image & Video Generation
    • Benchmarks & Evaluation

    UltraText Bench evaluates dense bilingual visual text rendering in image generation

    UltraText Bench is a bilingual benchmark with 432 prompts across 24 scene categories and three difficulty levels, assessing image generators on dense visual text reproduction in English and Chinese using a vision-language model for multi-dimensional scoring.

    Paper: UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation · Deyuan Liu, Yihao Hu, Jingxuan Zhang and 13 others

    153 upvotes on Hugging Face10 Oct 20266 min readRead the article

    • AI Agents
    • Benchmarks & Evaluation

    Learn2Play Bench tests LLM agents' ability to learn novel rules from interaction

    Learn2Play Bench introduces text-based games with hidden, unfamiliar rules to evaluate how well LLM agents learn from experience without weight updates. Experiments show raw memory retention aids learning, humans outperform agents by exploring more, and agent harnesses impact performance and cost.

    Paper: Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments? · Yibo Li, Jinhang Qiu, Zhi Zheng and 4 others

    108 upvotes on Hugging Face9 Oct 20266 min readRead the article

    • Benchmarks & Evaluation
    • AI Agents

    RealCompanion benchmarks AI understanding of real long-term human conversations

    RealCompanion releases ten real AI-human conversations spanning up to 120 days, with detailed annotations linking replies to past messages. It reveals that memory use is rare but often requires distant context, and current AI systems struggle to detect when to recall past information.

    Paper: RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations · Arman Behnam, Sunglyoung Kim, Liangwei Yang

    262 upvotes on Hugging Face5 Oct 20265 min readRead the article