To solve the problems of blurred edge information and poor accuracy of small targets feature extraction when the lightweight semantic segmentation algorithm is applied to the UAV high-resolution traffic scenes image segmentation, a lightweight semantic segmentation algorithm combining attention mechanism and ghost feature mapping is proposed. Firstly, the hybrid attention module is embedded in the semantic branch 8-fold and 16-fold down-sampling process of the BiSeNet V2 to redistribute the weights of the deep feature maps and enhance the local key feature extraction ability. Then the ghost feature mapping unit is used to optimize the traditional convolution layers to further reduce the computational cost. Finally, the dynamic threshold loss function is applied to supervise the training, adjusting the training weights of the high-loss difficult samples. Using the UAVid dataset to train and test the improved algorithm, it is found that the mIoU is 52. 7%, which is 7. 8% higher than the BiSeNet V2. When the input images size is 1 280×736, the inference speed can reach 73. 6 FPS, meeting the real-time segmentation requirements. The results show that the algorithm can be well adapted to complex traffic scenes, and can effectively improve the problems of blurred edge information and poor accuracy of small objects.