To solve the problem of serious spectrum performance loss of hybrid precoding in millimeter wave multiple input multiple output (MIMO) systems, a hybrid precoding algorithm based on alternating optimization orthogonal matching pursuit is proposed in this paper. Firstly, an analog precoding matrix under constant mode constraint is used to determine the good initial connection state between the stationary phase displacement phaser and the antenna to improve the iterative convergence rate. Then, the optimal candidate simulation precoding matrix is constructed according to the connection state to solve the global optimal index vector. Finally, the digital precoding matrix composed of the optimal index vector is fed back to the dynamic network, which can dynamically optimize and update the connection state of the phase shifter and the antenna array alternately. At the same time, the proposed algorithm only needs a small number of fixed phase shifters to achieve a good balance between spectrum performance and complexity. Simulation results show that compared with other existing algorithms, the proposed algorithm has higher spectral efficiency, higher iterative convergence rate and lower complexity, especially when the number of RF links is greater than the number of data streams, the improvement of spectral efficiency is more significant.