Aiming at discontinuous object edge segmentation in high-resolution remote sensing image semantic segmentation and low accuracy of small object segmentation, this paper proposes a remote sensing image segmentation algorithm based on improved DeeplabV3+. The algorithm first adopts distraction network called ResNeSt instead of the DeeplabV3+ original backbone network Xeception to extract richer deep semantic information, thereby improving the accuracy of image segmentation; secondly, the Coordinate Attention (CA) mechanism is introduced to effectively obtain more accurate target location information of segmentation to make the segmentation target edge more continuous; finally, the cascade feature fusion method (CFF) is adopted in the decoding layer to improve the semantic information representation ability of the network. The experimental results show that the algorithm has a high mIoU of 97.07% on the high-definition remote sensing image dataset of a city in southern China, which is 3.39% higher than that of the original model and a reflection of better utilization of image semantic feature information. This provides a new way of thinking for remote sensing image semantic information.