Tuesday, 6 October 2026 · 6 min read
In-Distribution Forcing stabilizes long video generation by controlling KV cache provenance
Drifting in long autoregressive video diffusion arises from out-of-distribution KV cache entries due to unseen provenance during rollout. In-Distribution Forcing aligns KV caching and conditioning with training, enabling stable minute-scale video generation and reducing drift.
Based on the paper In-Distribution Forcing for Long Video Generation at Test Time by Jeongwoo Shin, Youngyoon Choi, Sangwoo Jo, Hyunmog Kim and 4 others
Challenges in long-horizon autoregressive video diffusion and drifting
Autoregressive video diffusion models generate videos chunk-by-chunk, conditioning each new chunk on cached key-value (KV) representations of previous chunks. While these models produce high-quality short clips, extending generation to long videos causes drifting: progressive degradation where colors, textures, and motion dynamics shift or decay.
Existing approaches to mitigate drifting focus on KV conditioning—modifying or selecting which cached KV entries the model attends to during generation. However, these methods assume the cached KV entries themselves remain in-distribution, an assumption that breaks beyond the training horizon.
The paper identifies a new problem called the KV-provenance problem: during rollout beyond the training horizon, KV entries are cached under unseen contexts (provenance), making them out-of-distribution (OOD). This OOD caching causes errors to accumulate and drifting to worsen, and no conditioning strategy alone can fix this source of drift.
Sign in to read the whole paper
You have read the opening. The rest is free with a Bezen account, and no card is needed.
In the rest of this paper
- How the method works, step by step, with a flow chart
- Three figures from the paper, and how to read them
- How it was tested, what was found, and where it falls short