Abstract:To address the problem of direction-of-arrival (DOA) estimation in environments with mixed Alpha-stable distributed noise and Gaussian colored noise, a forward prediction and backtracking orthogonal matching pursuit algorithm based on fractional-order cumulants (FOC-LABOMP) is proposed under a coprime array framework. First, a signal reception model is constructed using the coprime array, where the difference co-array formed by sensor spacings is utilized to generate a virtual array. By introducing a hole-filling interpolation technique, the effective aperture of the physical array is extended to the continuous region of the virtual array, thereby enhancing the degrees of freedom and improving angular resolution. Second, the semi-invariant property of fractional-order cumulants is leveraged to effectively suppress the interference from both Gaussian colored noise and Alpha-stable noise. Furthermore, the proposed method incorporates a forward prediction and backtracking orthogonal matching pursuit algorithm, which evaluates the correlation of atoms via inner products and predicts their performance in future iterations to select the optimal atom. A backtracking strategy is employed to improve the accuracy of sparse recovery, ultimately yielding the estimated DOA values. The effectiveness of the proposed algorithm was validated through computer simulation experiments. Under mixed noise conditions consisting of Alpha-stable distribution and colored Gaussian noise, when the mixed signal-to-noise ratio (SNR) is 0 dB, the root mean square error of the proposed algorithm for DOA estimation is 0.536 8 °, which improves the accuracy by 41.45% compared to the PFLOM-MUSIC algorithm. The simulation results fully demonstrate that the proposed algorithm can achieve high-accuracy DOA estimation under mixed noise conditions.