Aiming at the problems of low detection accuracy and slow sorting speed of coal gangue sorting tasks in industrial scenarios, a coal and gangue target detection algorithm based on improved YOLOv5s is proposed. A lightweight attention mechanism CBAM is added to convolutional layer of the backbone network to improve the ability of target feature expression in complex cinder environment. Secondly, the BIFPN structure is added to the feature fusion layer. The bidirectional cross-scale connection and weighted fusion are carried out in the BIFPN structure to strengthen the feature information of shallow layer of coal gangue and the location information of high-rise coal gangue, and solve the problem that the color and texture of coal gangue are similar and difficult to classify; Finally, on the basis of the original algorithm DIoU, the aspect ratio of the bounding box is added to improve the accuracy of the inspection box detection. The proposed method is tested by using 10 000 coal gangue images collected in an industrial production environment as a dataset. Experimental results show that in comparison with YOLOv5s model before the improvement, on the premise that the detection speed remains basically unchanging, average precision mAP_0.5 of the improved algorithm reaches 93.3%, and average detection precision is increased by 5.1%, which realizes the requirements for target detection of coal gangue.