In the process of unmanned aerial vehicle (UAV) return without a reference map, the scene matching between the real-time map and the waypoint is the key to the success of the UAV return. In order to improve the real-time and robustness of scene matching, a UAV scene matching algorithm based on FAST corner detection and FREAK descriptor is proposed. Firstly, in order to improve the shortcomings of FAST corner detection method such as no scale invariance and redundant feature points, a multi-scale gridded feature detection method based on FAST corner is proposed. Next, the FREAK binary descriptor is simplified to improve the matching speed. Then, the K-nearest neighbor ratio method and RANSAC method are used for the initial and fine matching of the features, and a positioning model is established to obtain the actual distance between the waypoint and the current position of the UAV and orientation information. Finally, experiments are performed to verify the performance of the algorithm. The deviation of the positioning direction of the proposed algorithm is within 1 °, and the deviation of the image plane distance is stable within 0. 6 pixels, the running time is 0. 43 s, which is much shorter than the processing time of SIFT and SURF algorithms. In the case of conditions such as scale transformation and noise, compared with algorithms such as SIFT and SURF, the proposed algorithm has achieved a better correct matching rate and better robustness. The experimental results show that the proposed algorithm is robust and fast, especially in perspective transformation, it is more suitable for UAV vision-assisted navigation.