Ultrasonic welding technology is widely utilized in industrial manufacturing, however, factors such as welding parameters, equipment conditions, and operational techniques often lead to diverse welding defects. To enhance welding efficiency, this study proposes a lightweight deep-learning-based network, ultrasonic welding defect detection network, for ultrasonic welding surface defect detection. First, to address the limitations of conventional convolutional networks, which are often insensitive to fine details and prone to losing critical small-scale defect features in welding defect detection, this study introduces a novel stepwise attention convolution module. The SA-Conv architecture enhances the model’s ability to perceive defect features while reducing computational overhead. Second, to tackle the challenge of extracting complex welding defect features, this study designs a defect feature extraction network incorporating a deformable convolutional network module and a welding defect feature extraction module based on deformable convolution and SA-Conv. This network significantly improves defect representation in complex backgrounds, enabling the effective extraction of welding defect features with varying shapes and intricate characteristics. Finally, quantitative and qualitative experimental analyses demonstrate that UWD-Net achieves superior detection performance on both a self-constructed welding defect dataset and the publicly available NEU-DET dataset. On the self-constructed dataset, UWD-Net achieves an F1-score of 0.952 and a mAP@0.5 of 93.6%, while on the NEU-DET dataset, it attains an F1-score of 0.710 and a mAP@0.5 of 78.6%, outperforming other benchmark algorithms. Furthermore, UWD-Net has a lightweight model size of only 1.818×106 parameters and achieves an FPS of 145.80, effectively balancing detection accuracy and inference speed. These characteristics make UWD-Net well-suited for real-time defect detection and deployment in industrial applications.