StreamVibe enables proactive video agents to infer and adapt to evolving user intents

Vision-language models are moving from passive offline understanding to active, real-time assistance in streaming video scenarios. However, existing methods mostly respond to explicit user requests and treat interactions as isolated single turns, which limits their usefulness in real-world settings.

Real-world user intents are often implicit, requiring the agent to infer goals from ongoing observations rather than explicit commands. Additionally, the interaction state changes as tasks progress, so the agent must track task-relevant state to avoid outdated or redundant responses.

User intents also evolve dynamically during interaction, with new goals, revisions, or interruptions. Existing models generally fail to update which intents remain relevant, making them less effective for sustained, intent-aware assistance. Addressing these challenges requires continuous inference and updating of user intent alongside environment and task state tracking.