In order to solve the semiconductor manufacturing defect detection with low efficiency, the error rate is high, the result is not stable, imaging accuracy is low and cannot accurately detect the problem such as different kinds of defects. In this paper, by using a custom CCD industrial camera with a high ratio of optical microscope scan images on the surface of the wafer, combined with the improved YOLOv4 algorithm, a high precision wafer defect detection method based on deep learning is implemented. Experimental results show that the proposed model can identify different kinds of silicon carbide wafer defects under various complex conditions and has good robustness. The average accuracy of defect identification is 99. 24%, which is about 10. 08% and 1. 92% higher than that of YOLOV4-Tiny and original YOLOv4, respectively. Compared with the Halcon-based method and OpenCV template matching method, the average recognition time of defects per graph reaches 0. 028 3 s, which is about 93. 42% and 90. 52% higher than other conventional wafer defect detection methods and has realized stable operation in independently designed verification systems and application platform.