In view of the fact that permanent magnet synchronous motor (PMSM) will be affected by nonlinear factors such as parameter perturbation and external uncertain interference in actual operation, which will lead to the decline of motor control performance and position tracking accuracy. To improve the control performance of permanent magnet synchronous motor(PMSM) under nonlinear factors, a nonlinear controller is designed by combining sliding mode control (SMC) with Backstepping Control, the reaching law in backstepping sliding mode control is improved, and a new sliding mode reaching law is designed by using a hyperbolic tangent function instead of symbolic function in traditional exponential reaching law. The nonlinear disturbance observer (NDOB) is used to observe and estimate the disturbance and compensate the disturbance, which is combined with the designed nonlinear control to track the motor position. Finally, the improved backstepping sliding mode control module and nonlinear disturbance observer module are built by using Matlab/Simulink. The results show that this method improves the motor position control accuracy and tracking response speed to a certain extent, reduces the position tracking error, and enhances the anti-interference ability of the system.