Aiming at the problem of low accuracy and overfitting in the unbalanced data of molecular pump of EAST all-superconducting tokamak device, a method of time-frequency analysis and improved LightGBM algorithm is proposed. Firstly, the normal and fault vibration data are collected by the molecular pump experimental platform. Then, extract the time and frequency domain features. Moreover, the cost-sensitive LightGBM fault detection framework was established by optimizing the misclassification cost function. Finally, the obtained features are used as the input of the cost-sensitive LightGBM algorithm for molecular pump fault detection. The experimental results show that the fault detection accuracy is 99. 4%. Meanwhile, the proposed method can consistently outperform traditional classifiers and LightGBM algorithms. This method can effectively solve the problem of overfitting and realize the detection of molecular pump fault with high accuracy.