At present, garbage classification has been widely concerned by the government and society. Facing the demand for real-time and accurate judgment of waste classification in the sorting process, a Yolo_ES target detection algorithm was proposed. The algorithm takes Yolov4 as the basic network. Firstly, EfficientNet is used as the backbone feature extraction network to realize the lightweight of the algorithm; secondly, the MBConv module is reconstructed by the attention mechanism ECA to filter out the highquality information, enhance the feature extraction ability of the model and reduce the number of parameters; at the same time, aiming at the problem that it is easy to lose detailed information in the max-pooling, the SoftPool is used to replace the MaxPool layer in the SPP module to retain more fine-grained feature information. The experiments are conducted on the self-made HPU_WASTE garbage classification dataset, and the results show that compared with the Yolov4 basic network, Yolo_ES model increases the map from 91.81% to 96.06%, and the model size is compressed by 75.45%. Meanwhile, the processing time of each image is 58ms; Compared with other target detection networks, this model has better robustness and better detection performance.