The deviations from the reference track of airborne synthetic aperture radar (SAR) systems decrease the resolution in SAR imagery, and motion error estimation based on raw data is an important step for motion compensation. According to the research of motion error estimation, a new approach for estimation of motion errors based on regularized total least square (RTLS) is proposed. This new method fully considers the effects of the heteroscedasticity of phase estimation error in different range cells, the error of model coefficient matrix and the illconditioned problem in the motion error estimation. It combines weighted method, total least square with regularization method while in process to eliminate the heteroscedasticity and decrease the effects of coefficient matrix error and illconditioned problem. It also utilizes the Lcurve method to realize the automatic regularization parameter selection. The experimental results show that the proposed method is validated and accurate to motion estimation.