In remote sensing image target detection, the remote sensing image is usually arranged in any direction under the top view angle. This situation makes common detection algorithms have good detection results in natural scenes, but the detection results are often unsatisfactory in remote sensing images. Aiming at the problem of unsatisfactory detection in remote sensing scenes, this paper proposes a remote sensing image target detection algorithm (CSL-YOLOv5) based on the rotating target frame and attention mechanism based on the single-stage detection network YOLOv5. First of all, the original network feature extraction network (CSPDarknet53) was modified to increase the number of output feature maps and optimize the detection effect of the network on small targets. Then, an attention mechanism that combines the channel module and the spatial module is added to the residual block to enhance the expression effect of image features. At the same time, Focal loss is used to optimize the training effect, and the detection accuracy is improved on the basis of ensuring the detection speed. Finally, the long-side representation based on circular smooth labels is used to achieve the rotation of the target frame, and the effect of angle periodicity on training is solved by turning the regression problem into a classification problem. The experimental results show that the CSL-YOLOv5 algorithm proposed in this paper achieves a detection accuracy of 76.24mAP in the DOTA data set, which has a higher accuracy compared with the previous single-stage algorithm, and has an increase of 8.06% compared to the mAP of YOLOv5. The algorithm in this paper has high detection accuracy and good robustness in remote sensing scenarios.