At this stage, the image deep learning algorithm cannot detect the chronological process problem. In this paper, the artificial assembly process of the mountain board assembly of knitting machinery is studied, and the MS-RetinaNet object detection algorithm is proposed. Using the idea of natural language processing for reference, the Swing-Transformer structure is introduced to retain the hierarchy of CNN structure, make up for the lack of high-level semantic information fusion in CNN structure, and enhance the ability to learn overall and details. The improved GIoU Loss is used to increase the judgment factor formula, mitigate the impact of loss calculation degradation, and optimize the regression effect of the bounding box. According to the multi-scale target parameters, the best anchor frame ratio is adopted to improve the recall rate and detection accuracy. The chronological detector is designed to enable the algorithm to distinguish the sequence and logical relationship of the target. The experimental results show that the algorithm AP can reach 90. 3%, which is more than 2% higher than the current mainstream algorithm. The detection speed of a single image is about 46 ms, meeting the chronological detection requirements of the process flow, and the overall performance is superior.