In order to improve the efficiency of multi-class container damage detection in complex yard environment, a lightweight container damage detection method based on improved RETINEX is proposed. The method mainly consists of two parts, image preprocessing and lightweight target detection: In image preprocessing stage, the luminance channel component is introduced and optimized, and when applying the multi-scale Retinex processing method to it, a bilateral filter is used instead of the traditional Gaussian filter to retain the edge details of the original object; the value domain conversion function is improved to reduce the loss of image data; and a color protection is obtained through the calculation of the color balancing strategy factor, which is multiplied with the pixel points of each channel of the original RGB image to get the enhanced image. In target detection stage, MobileNetv3, a lightweight network with improved attention mechanism, is introduced into the YOLOv5 backbone network to construct a text-based target detection network, so as to validate the port container images. The experimental results show that the method helps the target detection network to extract richer feature information in complex port environments such as low illumination, and the average detection accuracy of multiple container damage types is improved by 1.4% to 95.1%, and the model volume is only 20.5 MB, which is a significant advantage compared with multiple mainstream detection algorithms, and it can satisfy the actual detection needs of port containers, proving the effectiveness of the method in this paper.