To achieve person recognition in water environment, a person category identification algorithm based on vision sensors on unmanned surface ship (USV) is proposed. Firstly, base on the data acquisition and model update workflow, a person category dataset of 39 959 pictures and 7 categories is created after data cleaning and labeling on original videos. Secondly, YOLO v5, the mainstream object detection network in the field of deep learning method, is practiced, and an improved person category identification algorithm based on YOLO v5 is proposed according to the characteristics of water environment scenes. Thirdly, the algorithm is deployed to the edge computing platform to realize the real-time use of the algorithm on the unmanned ship. The algorithm achieves an average accuracy of 86% on our dataset and achieves real-time performance of processing 38 frames per second with accurate person recognition in the unmanned ship test.