Do LVLMs Truly Understand Video Anomalies? Revealing Hallucination via Co-Occurrence Patterns

Large Vision-Language Models (LVLMs) pretrained on large-scale multimodal data have shown promising capabilities in Video Anomaly Detection (VAD). However, their ability to reason about abnormal events based on scene semantics remains underexplored. In this paper, we investigate LVLMs’ behavior in VAD from a visual-textual co-occurrence perspective, focusing on whether their decisions are driven by statistical shortcuts between visual instances and textual phrases. By analyzing visual-textual co-occurrence in pretraining data and conducting experiments under different data settings, we reveal a hallucination phenomenon: LVLMs tend to rely on co-occurrence patterns between visual instances and textual phrases associated with either normality or abnormality, leading to incorrect predictions when these high-frequency objects appear in semantically mismatched contexts. To address this issue, we propose VAD-DPO, a direct preference optimization method supervised with counter-example pairs. By constructing visually similar but semantically contrasting video clips, VAD-DPO encourages the model to align its predictions with the semantics of scene rather than relying on co-occurrence patterns. Extensive experiments on six benchmark datasets demonstrate the effectiveness of VAD-DPO in enhancing both anomaly detection and reasoning performance, particularly in scene-dependent scenarios.

Authors

• Menghao Zhang
• Huazheng Wang Oregon State University
• Pengfei Ren State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications
• Kangheng Lin
• Qi Qi
• Haifeng Sun Beijing University of Posts and Telecommunications
• Zirui Zhuang Beijing University of Posts and Telecommunications
• Lei Zhang
• Jianxin Liao Beijing University of Posts and Telecommunications
• Jingyu Wang Beijing University of Posts and Telecommunications Pengcheng Laboratory

Keywords

No keywords are indexed for this paper.