In order to lighten the rust image segmentation network model and eliminate the interference of nonsingle feature background and similar feature backgrounds such as rust liquid, this paper replaces the encoded part of the U-Net network model with the MobilenetV3_large network, imports the pre-trained weights of the MobilenetV3_large network based on the ImageNet dataset, and replaces the ordinary convolution of the decoded part of the U-Net network model with a deep separable residual convolution. And add the attention-oriented AG module and the Dropout mechanism in the process of upsampling. Experimental results demonstrate that the improved U-Net network model designed in this paper exhibits significant advantages in rust image segmentation under non-uniform feature background and similar feature background interference such as rust liquids. The model size is reduced by 81.18% compared to the original U-Net network model, resulting in a decrease of floating point calculations by 98.34%. Additionally, the detection efficiency has improved by 3.27 times, increasing from less than 6 frames/s to 19 frames/s. While the network model is lightweight, the accuracy of the network model is 95.54%, which is 5.04% higher than the original U_Net network model.