In this paper, a tracking algorithm based on CAMShift which combined with SURF feature matching and Kalman filter is proposed to deal with the problems in traditional CAMShift algorithm, such as tracking failure under color interference or occlusion. The algorithm calculates the Bhattacharyya coefficient of integrated histogram composed of chroma feature and gradient direction feature between candidate target and template target as judging basis, it uses CAMShift algorithm. As the coefficient more than the threshold, SURF algorithm will be used to match the search window and the tracking result of the previous frame, then recalculate the target’s size and position by the matching result. To avoid tracking failure by fastmoving of target and reduce the computation of SURF matching, the center position of moving target in the next frame will be predicted by Kalman predictor. The experimental results show that the new algorithm can achieve stable tracking object against complex backgrounds, color interference or occlusion, and have higher tracking speed than CAMShift algorithm combined with SURF.