With the rapid and comprehensive development of China's economy and the improvement of people's living standards, China’s demand for transportation is increasing. In order to meet the higher requirements of locomotive operation safety, a visual algorithm of the advanced driver assistance system of trains based on single-stage instance segmentation in railway scene is proposed. The algorithm model is optimized for detecting the characteristics of multiple overlaps of objects in railway scenes. The accuracy of the model is improved by improving the Backbone network and multi-scale fusion method. The model is accelerated by TensorRT semi-precision acceleration and CUDA code refactoring. The performance evaluation and comparative test of this method and other methods are carried out. Finally, this method achieves 71.2MAP and 108ms on the embedded platform Xavier. High-precision detection of the surrounding environment of the train under vehicle deployment is realized.