Because of the great difference of sensor imaging, the registration of high-resolution remote sensing image is faced with more prominent local deformation problem.In particular, the relative parallax offset of high-rise objects in the city is more significant, which leads to serious nonlinear error in spatial transformation.Therefore, this paper proposes a registration method for multi-source high resolution remote-sensing image based on high-rise objects.Firstly, through shadow detection and image segmentation, the high-rise objects are screened.On this basis, a threshold adaptive feature point extraction strategy based on phase consistency is proposed, in order to improve the number of feature points in high-rise objects and their overall distribution uniformity.Secondly, the distance of feature vector weighted by shadow area is introduced to eliminate the interference of shadow on feature point matching.Finally, an adaptive penalty factor of transformation error is designed to reduce the influence of high-rise objects’ spatial variation on affine equation.Through the registration experiments on groups of multi-source high resolution remote sensing images, it is found that the registration accuracy and root mean square error of the proposed method can reach 88.9% and 1.481 respectively.