In response to the safety risks associated with abnormal passenger behavior in elevators, an enhanced anomaly detection model, YOLO_LP, based on YOLOv11, is proposed. First, a novel feature extraction component, CSP-PTM, is incorporated into the backbone network. This component enables powerful local and global feature extraction, significantly improving the model’s detection accuracy. Next, a contextual information fusion module is introduced to enhance the feature pyramid network. This approach reorganizes feature information through a weighting mechanism, effectively improving the discriminative capability of the feature maps. Additionally, the wise intersection over union loss （WIoU） function is employed to address class and size imbalances, further enhancing the model’s accuracy and convergence speed. Finally, a newly designed LDH detection head replaces the original, resulting in a lightweight network model. Experimental results demonstrate that the improved model achieves an accuracy of 90.4% in detecting abnormal passenger behavior in elevators, 3.5% higher than the baseline model. Furthermore, compared to the YOLOv11n model, it shows improvements of 2.9% and 2.1% in mAP@0.5 and mAP@0.5:0.95, respectively, while reducing the number of parameters and computational load by 10% and 17%, respectively. These findings highlight the superior performance of the YOLO_LP model, which meets the accuracy and speed requirements for abnormal behavior detection in elevator cabins.