Abstract:Instance segmentation is an important research direction in the field of computer vision, but the existence of the occlusion problem still prevents this task from being fully explored. To address the poor segmentation detection of occluded objects by current algorithms, which are prone to the problems of misdetection and omission, a novel duplex residual network (DRNet) is proposed, combining the EIoU occluded target segmentation model with the Mask R-CNN framework. First, DRNet is proposed to replace the original ResNet network, using fewer BN and ReLU layers to replace the traditional Conv-BN-ReLU structure, utilizing the conventional convolution and depth-separable convolution serial connection to enhance the image sensory field features, and mitigating the degradation problem of the network with the increase of the depth by the hopping connection. Second, the CEIoU NMS algorithm is used instead of the original NMS algorithm to effectively deal with the overlapping bounding box suppression problem through the clustering idea, and the introduction of the EIoU evaluation index increases the bounding box geometric information, which more accurately describes the degree of similarity between the bounding boxes, and reduces the network’s erroneous suppression of the bounding boxes of the occluded objects. Finally, the EIoU loss is used to replace the original Smooth L1 loss to accelerate the network convergence speed and improve the bounding box detection accuracy. In this paper, we first conduct pre-training on the public COCO 2017 dataset and experiments on different degrees of occlusion datasets, and the results show that compared with the original network, the proposed segmentation algorithm improves the Box AP and Mask AP by 1.7% and 1.3% on the COCO 2017 dataset, respectively; and both the bounding-box detection accuracy of the occluded object and the mask segmentation accuracy on the occlusion dataset are significantly improved on the occlusion dataset, confirming the effectiveness of the method for occluded object segmentation.