Aiming at the fast and accurate requirements of the image processing for the feasible domain and obstacle segmentation system of unmanned surface vessels (USV), an algorithm for fast segmentation of images on water according to the on-board vision sensor of unmanned surface vessels (USV) is studied. Firstly, the experimental images were collected through multiple experiments, and the original database was constructed through data cleaning, image de-duplication, and manual screening. The feasible region and obstacle segmentation data set of the unmanned ship were constructed using the Human-in-the-loop annotation method, with a total of 5 620 images and 25 875 tags. Secondly, it practices the mainstream semantic segmentation methods based on deep learning, including FCN, DeeplabV3 Plus, U-Net. Finally, a fast segmentation network DeeplabV3-CSPNet based on improved DeeplabV3 Plus is proposed according to the characteristics of water images and the requirements of fast segmentation. The results of the network learning experiment, offline navigation experiment, and model deployment results show that the DeeplabV3-CSPNet algorithm achieves a fast and accurate segmentation with an average accuracy of 84. 17% and an operation speed of 49. 26 fps, which can reach 45. 45 fps on the edge computing platform.