Thursday, 22 October 2026 · 6 min read
SpatialScout benchmark tests VLM agents on active 3D object mapping in indoor scenes
SpatialScout evaluates vision-language models on autonomous exploration and 3D object localization in 80 indoor scenes with 3,140 targets. The best model achieves 41.71% mean IoU, revealing challenges in precise, complete metric mapping through active perception.
Based on the paper SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes? by Zhaoxi Zhang, Hao Liang, Linxuan Wu, Yiming Wu and 6 others
Challenges in evaluating spatial intelligence through active 3D mapping
Spatial intelligence involves an agent actively gathering visual information and integrating it into a coherent 3D representation of its environment. This requires resolving occlusions, perspective ambiguities, and associating observations across multiple views to infer object locations and spatial relationships.
Existing benchmarks mostly focus on spatial question answering (QA) tasks, which test models' ability to answer questions about spatial relations or counts from static images or videos. However, these QA tasks do not require agents to autonomously explore or build explicit spatial maps, and can be solved partially through language priors or guessing.
To truly assess spatial intelligence, a benchmark must require agents to actively select viewpoints, acquire new evidence, associate observations across views, and construct explicit metric maps of object locations. This involves a closed loop of perception, reasoning, and decision-making over multiple interaction steps, which is more representative of real-world spatial understanding demands.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- The key formula from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short