The object of remote sensing image has the characteristics of complex background and changeable direction. The process of object detection in remote sensing image using traditional methods is complex and time-consuming, with low accuracy and high rate of missed detection. To solve the above problems, we propose an improved YOLOv5AC algorithm. This algorithm bases on the YOLOv5s model. First, an asymmetric convolution structure is built in the original Backbone to enhance the robustness of the model to flipped and rotated targets; Secondly, coordinate attention mechanism is introduced into C3 module of backbone network to improve feature extraction capability, and Acon (Activate Or Not) adaptive activation function is used for activation; Finally, we use CIOU as the location loss function to improve the positioning accuracy of the model. The improved YOLOv5-AC model was tested on NWPU VHR-10 and RSOD datasets, and the average accuracy reached 94.0% and 94.5%, respectively, 1.8% and 2.3% higher than the original YOLOv5s, which effectively improved the object detection accuracy of remote sensing images.