Road damage detection is an important basic link in the process of road maintenance. Traditional road damage detection methods have the defects of high detection cost and low efficiency. In order to accurately and quickly detect road damage, an improved road damage detection model YOLO-C-α based on YOLOv5 is proposed. By introducing the attention mechanism CBAM module, the feature extraction and feature fusion capabilities of the detection model are improved, and the problem of missed detection of small targets with road damage is improved; the α-IoU loss function is introduced to replace the CIOU loss function of the original network to reduce the regression loss of the prediction frame, to improve the positioning accuracy of the prediction box. Based on the RDD2020 road damage detection data set, a comparative experiment was carried out. The results showed that the average accuracy of the YOLO-C-α model reached 60.3%, which was 1.4% higher than the average accuracy of the original model. Its F1 value was 60.2, compared with the original model. It is improved by 1%, and has high detection performance for pavement damage under different weather conditions. The detection speed of each image in the experimental environment is 6.3 ms, and the model size is 40.6 Mb. The results show that the improved algorithm based on YOLOv5m has strong anti-interference ability and can more accurately detect road damage targets under various weather conditions, which can provide a reference for real-time road damage detection and intelligent road maintenance.