VA-Judger: Reward Modeling for Joint Video-Audio Generation from Human Preference Feedback

Summary

Reward modeling for joint video-audio generation typically combines metrics for audio quality, visual fidelity, and synchronization, which evaluate perceptual dimensions separately and fail to capture the overall semantic and temporal coherence among text, video, and audio, encouraging reward hacking. To address this, the authors construct a large-scale human-preference dataset VAPref-10K with 9K prompts and 10.3K fine-grained paired comparisons from open-source generation models. They introduce VA-Judger-Bench for in-domain and out-of-domain evaluation, and propose VA-Judger, a chain-of-thought omni-reward model. VA-Judger first learns from pairs with clear quality gaps, then distills reliable preference explanations for harder near-quality comparisons, and finally performs dimension-wise reinforcement learning to decompose human feedback into individual quality dimensions for denser rewards. Experiments show VA-Judger outperforms metric baselines in predicting human preferences on both in-domain and out-of-domain evaluations, and using its rewards for post-training improves generation quality.