In order to avoid the damage of underground cables and improve the early warning ability of the vibration monitoring system against external force damage, an identification method of external damage vibration signal based on time-frequency spectrum and adaptive dynamic inertia weight PSO-CNN is proposed. Firstly, 3000 groups of external vibration signals obtained by the vibration sensing system are converted into time-frequency spectrum data sets. In the image preprocessing stage, histogram equalization and 2D-PCA algorithm are used to enhance the characteristics of gray image and reduce the dimensions of image data; Then, 70% of the image dataset is taken as the training set of the CNN model, and the adaptive dynamic inertia weight particle swarm optimization (PSO) algorithm is introduced in the network training process to iteratively optimize the relevant parameters of the convolution layer and pooling layer of the CNN model, so as to obtain the optimized PSO-CNN classification model; Finally, the recognition performance of the optimized PSO-CNN model is verified by using test set image data, and which is compared with other classification models. The results show that the recognition accuracy of the proposed method for six common external damage vibration signals reaches 98.33%, and the average recognition time of each image is only 0.24s. Compared with other classification algorithms, the proposed method has higher classification accuracy and faster recognition speed, which provides a feasible scheme for quickly and accurately identifying the types of external damage events.