Violence Detection of Weakly Supervised Surveillance in the Real World Through Transformer-Based Attention Multiple Instance Learning

Abstract

While Automated violence detection in surveillance videos is a key requirement of smart city security, it remains challenging due to several factors, such as temporal sparsity, background clutter, and lack of frame-level annotation. To address the above challenges, this paper proposes a weakly supervised violence detection framework that integrates a SwinBase (Swin-B) Vision Transformer with an attention-gated Multiple Instance Learning (MIL) mechanism. In the proposed approach, each video is treated as a bag of uniformly sampled frames. To process the video classification, we first extract frames. For feature extraction, we trained it on a frozen Swin-B backbone, which is pre-trained on ImageNet. Now, to extract meaningful information from the frame, we use attention-gated MIL pooling, which is used to aggregate informative instances from the video and suppress irrelevant background frames. This enables effective video-level classification. The framework is designed to be computationally efficient and suitable for largescale surveillance scenarios. We chose three benchmark datasets, namely Hockey Fight, RWF-2000, and Real-Life Violence Situations (RLVS). Our proposed method achieves near-perfect performance on the Hockey Fight dataset and demonstrates strong generalization on challenging real-world datasets, with ROC-AUC values exceeding 0.97. The results confirm that by combining (SWIN-B) transformer-based feature extraction with attention-gated MIL, a robust and scalable solution for videobased violence detection in both controlled and uncontrolled environments, which are related to surveillance.