In order to solve the problem such as need the strict optical calibration in current optical image encryption algorithm, the optical image encryption algorithm based on circular harmonic component expansion and Gyrator transform domain phase retrieval was proposed in this paper. Firstly, Gyrator transform spectrum was formed by introducing Gyrator transform to deal with the plain. Then the spectrum of Gyrator transform is divided into zero order harmonic components and nonzero order harmonic components based on offaxis circular harmonic component expansion mechanism. The cipher was obtained by introducing the spherical phase factor to modulate the nonzero order harmonic component. Finally, the complex distribution was obtained by using the retrieval algorithm of Gyrator transform to take the zero order and nonzero order amplitude as the constraints condition of input plain and output cipher to finish the image encryption. The experimental results show that this algorithm has higher security and anti filtering robustness with stronger antinoise attack and antishear attack ability. This proposed algorithm can better protect the image in the network security transmission with good practicality.