Wafer defect classification is a critical task in the chip manufacturing process, as accurately identifying various types of defect patterns is essential for promptly locating and resolving manufacturing issues. To overcome the limitations of insufficient multi-scale feature representation and low efficiency in cross-level information fusion, this paper proposes a novel multi-level feature fusion network based on convolutional neural networks (CNN) and Transformers, named CTM-Net. The network adopts a dual-branch parallel architecture: the CNN branch focuses on extracting local morphological structures and fine-grained defect details, while the Transformer branch employs self-attention mechanisms to capture global contextual dependencies and structural information. Based on these complementary representations, a multi-level feature interaction module is further designed to adaptively integrate and enhance cross-scale features, effectively compensating for the limitations of single-branch representations and improving the model’s discriminative capability. Experiments conducted on the publicly available WM-811K dataset show that CTM-Net achieves an accuracy of 99.31%, outperforming five representative state-of-the-art wafer defect classification approaches. Notably, the proposed model demonstrates superior recognition accuracy and stability in critical defect categories such as Edge-Loc, Edge-Ring, and Loc, highlighting the effectiveness and advantages of combining CNN and Transformer architectures for multi-level feature fusion in wafer defect classification tasks.