Thursday, 5 November 2026 · 7 min read
TurnWise improves streaming video RL by assigning credit at each decision turn
TurnWise refines reinforcement learning for streaming video understanding by assigning turn-level advantages using silence-completed prefix scoring and averaging losses per turn, outperforming prior methods on three benchmarks without extra critics or rollouts.
Based on the paper Turns Matter: Advancing Streaming Video RL via Turn-Level Credit by Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao and 19 others
Challenges in streaming video understanding and RL credit assignment
Streaming video understanding requires models to decide repeatedly whether to respond or wait for more input, under partial and evolving observations. Unlike offline video tasks where the entire video is available before answering, streaming models must balance timely, informative responses with silence to avoid redundancy.
Reinforcement learning (RL) has been applied to learn these response/wait decisions using trajectory-level rewards that assess overall interaction quality. Group Relative Policy Optimization (GRPO) is a recent method that assigns a single advantage score per trajectory to all decisions within it.
However, using a shared trajectory-level advantage for all turns causes problems: later decisions may inherit credit from earlier successes, reinforcing redundant replies, or be penalized due to earlier failures, discouraging useful follow-ups. This mismatch between trajectory-level feedback and turn-level decisions complicates learning effective streaming policies.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- The key formula from the paper, explained symbol by symbol
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short