In order to solve the problem of detecting the text information of traffic signs by the external environment perception system of unmanned vehicles, a two-stage method for detecting and recognizing the text information of traffic signs in an autonomous driving scenario is proposed, which realizes the refined collection of autonomous driving information. First, use the YOLO detector to detect traffic signs. At the same time, use the improved DB detection network in this article to detect the text in the scene. The intersection of the traffic sign detection results and the scene text detection results to get the text area to be recognized; finally, the lightweight CRNN network is used to treat Recognize area text for recognition. Use CSCT-1600 data set and MTWI-2018 data set for training and testing respectively. The experimental results show that the accuracy of the traffic sign information positioning algorithm is 94.95% when the recall rate is 92.98, and the recognition speed of the traffic sign information recognition algorithm is 25 frames when the F1 is 77.2%.