Surface defects on the skin of aircraft can affect its aerodynamic characteristics, and in severe cases, even compromise flight safety. To address the problem of low detection accuracy in detecting surface defects on aircraft skin, a method based on improved YOLOv5 is proposed for detecting four types of defects: cracks, corrosions, scratches, and strikes. The method first collects a dataset of aircraft skin surface defect images, and removes similar images using the MSSIM algorithm to improve the model training more efficient and generalizable. Then, the convolutional block attention module CBAM is integrated into the backbone section of yolov5 to enhance the feature extraction ability. Finally, the CSP_2 module is replaced with swin transformer block module in the neck section to further integrate global and local features of defects and improve defect detection accuracy. The experimental results show that the improved method has better detection performance, with accuracy, recall, and average precision of 88.29%, 87.13%, and 92.88% respectively, which are 3.28%, 3.04%, and 2.77% higher than YOLOv5s. The proposed method can provide technical reference for aircraft skin surface defect detection.