The quality inspection of metal parts is a labor-intensive work. Generally, fatigue test is used to evaluate the quality of metal parts and the basis for accurate evaluation of the quality of metal parts, which can guide the design of reasonable mechanical structure and effectively reduce safety accidents and economic losses. In order to quickly screen qualified metal parts with strong vibration resistance, an online detection algorithm of vibration fatigue crack based on computer vision is proposed for metal parts with rapid vibration in fatigue test. Firstly, the displacement and motion blur caused by vibration are eliminated based on the front and rear frame alignment method, and then the surface changes of parts are detected by the inter frame difference method. Finally, the crack shape and length parameters are obtained by the segmentation algorithm according to the texture and geometric characteristics of the crack. The experimental results show that compared with the crack detection algorithm under static conditions, the proposed algorithm can automatically obtain the location, length and shape of parts’ cracks without stopping the shaking table, and record the crack propagation process, which can greatly improve the work efficiency of vibration crack detection.