Ultrasonic defect detection is a mainstream means of defect recognition, and two-dimensional convolution neural network has always been the main technology in this field. Generally, attribute features are extracted from two-dimensional C-scan, D-scan and other images for recognition and classification. These studies mainly use two-dimensional convolution layer, which will consume a lot of resources. Among all kinds of defect identification methods, ultrasonic echo signal analysis is one of the most important and useful tools. In this study, features are extracted from the original time domain ultrasonic signals, firstly, the data are collected using the JPR-600C air-coupled ultrasonic nondestructive testing system from the laboratory, and then the one-dimensional t-sne network model is constructed and optimized by using different hyperparameters, t-sne visualization and other means. Finally, ultrasonic signal defect recognition and classification is realized. The experimental results show that the performance of the CNN model proposed in this paper is better, and the correct rate of defect identification is 97.57%, which is higher than other machine learning methods, which provides some assistance for the automation of defect recognition.