In the complex background environment, the tracking technique for the target of modelfree model is an important subject in the field of computer vision research. The difficulty lies in how to accurately and reasonably extract the keypoints of the set target in the complex background and the movement way. In this paper, a consistent feature points based on matching tracking algorithm, through longterm modelfree model of target tracking, the rigid target objects on the ground in the random velocity, through light change area, partial occlusion target area, object turning or rotating, acquisition device on the target scaling case stabilization and tracking problems. The test results show that the algorithm can achieve the stable tracking of the calibration target in the above circumstances, and it has a certain realtime performance.