The optimization design of diagnosis strategy is an important part in the process of testability design. Unreliable test factors seriously affect the optimization design process. This paper summarizes previous research results. Aiming at the problem that heuristic search algorithm is difficult to solve the problem of diagnosis strategy optimization under unreliable testing conditions, this paper proposes a diagnosis strategy optimization algorithm based on the essence ant system. This paper establishes a mathematical model for the optimization of the diagnosis strategy under unreliable conditions, and then constructs the optimized target with the cost of testing and the cost of error. Then, it uses the improved ant system algorithm to solve the problem. Finally, the algorithm is applied to an equipment for instance analysis. Compared with greedy algorithm and common ant colony algorithm, it shows the advantages of the algorithm in precision and convergence speed, and verifies the feasibility and effectiveness of the algorithm.