FlowLoss: Dynamic Flow-Conditioned Loss Strategy for Video Diffusion Models

Summary#

This paper studies how to improve motion consistency in video diffusion models. Instead of using a purely warping-based objective, it introduces FlowLoss, a loss that directly compares optical flow fields from generated and ground-truth videos.

The key idea is that flow supervision is not equally reliable throughout diffusion denoising. To handle that, the method uses a noise-aware weighting strategy so the flow signal contributes differently across timesteps.

Abstract#

Video Diffusion Models (VDMs) can generate high-quality videos, but often struggle with producing temporally coherent motion. Optical flow supervision is a promising approach to address this, with prior works commonly employing warping-based strategies that avoid explicit flow matching. In this work, we explore an alternative formulation, FlowLoss, which directly compares flow fields extracted from generated and ground-truth videos. To account for the unreliability of flow estimation under high-noise conditions in diffusion, we propose a noise-aware weighting scheme that modulates the flow loss across denoising steps. Experiments on robotic video datasets suggest that FlowLoss improves motion stability and accelerates convergence in early training stages. Our findings offer practical insights for incorporating motion-based supervision into noise-conditioned generative models.