To address the challenge of ensuring safety, efficiency, and smoothness in collision avoidance decisions for unmanned surface vessel in complex environments with dynamically changing obstacles, we propose a collision avoidance method that combines the dynamic window approach and DDPG algorithm. Firstly, in the traditional collision risk model, the distance to closest point of approach and the bearing angle at the closest point of encounter are added as evaluation factors to make the risk evaluation of the unmanned surface vessel more reasonable. Next, we design a local guidance method based on dynamic window approach, the reachable position of unmanned surface vessel by dynamic window approach is used as the local guide point, and the guide reward is added to increase the action reward near the guide point, so that DDPG algorithm can obtain more accurate updating direction in training. Finally, the method is tested in various obstacle environments. Experimental results show that compared to the traditional DDPG algorithm, the proposed method generates more reasonable, smoother and less risky paths. Additionally, it improves convergence speed by approximately 37.5%, verifying the effectiveness of the proposed method.