For the classification of metal defects, the mainstream classification methods represented by deep learning are mainly statistical learning methods based on large datasets. However, when applying deep learning, not only many quality labeled samples are needed, but also the result may suffer poor generalization. A classification approach with few samples is proposed, which embeds the hierarchical and concise knowledge of humanoid into deep learning. First, a CNN is built as the backbone of the classification model, and a humanoid learning module is designed, which uses the features of human classification to classify. To improve the generalization, robustness and better fusion effect of the model, a mathematical integration model based on logarithmic function is designed. The mathematical integration model in the module couples the outputs of backbone network and humanoid learning module by using the idea of integrated learning. The experimental results show that for the metal defect data of small training set and large test set, the classification performance and the amount of training parameters are better than the deep learning method. Humanoid learning module and mathematical integration model are embedded in different backbone, and good performance is achieved, which shows that the proposed method is suitable for various deep convolution neural networks.