In the process of coal transportation, there are often foreign bodies scratching or tearing the transportation belt, resulting in safety accidents such as coal outlet blockage. Therefore, it is necessary to identify and classify the foreign bodies on the coal conveyor belt in time, so as to carry out early warning, sorting and control to reduce the probability of accidents. Aiming at the problems of large amount of calculation parameters and low classification accuracy in most classification networks, a classification network of coal belt foreign bodies based on residual network is proposed. The network uses multiple small convolution layers instead of the 7×7 convolution of the first layer to enhance the capture ability of local features and adds BN layer and ReLU activation function to make the network converge faster and enhance the nonlinear ability of the network. In the residual block, the depthwise separable convolution is used instead of the ordinary convolution, which greatly reduces the parameter quantity and calculation amount of the network and speeds up the model inference. After adding the CBAM attention mechanism to the convolution layer in the residual block, the network′s ability to learn channel features and spatial features is enhanced, the influence of useless background information on the model is weakened, and the attention is focused on the coal belt area. The deep features are fused with some shallow features to improve the recognition rate of small target foreign bodies such as anchors. The accuracy of the network on the self-built mine data set reached 91.4%, which was 4.7% higher than that of the improved network. The recall rate reached 91.2%, which was 5.8% higher than that of the pre-improved network. The calculation amount was reduced by 20%, and the number of parameters was reduced by 31%. The results show that the constructed network has higher accuracy, lighter weight, faster training speed and stronger real-time performance.