To tackle the difficulty in deploying on the side devices of the ARM platform triggered by the huge computation amount and memory occupation of deep learningbased object detection algorithms, this paper presents a lightweight method based on ARM platform object detection. Innovatively, this research adds constraints to the scaling factor of the batched normalized layer and the convolution kernel parameter of the convolution layer in the network, performs sparse training, and uses the scaling factor and the convolution kernel parameter as two criteria for judging the importance of channels and thus pruning the unimportant channels. Furthermore, CBAM attention is adopted to achieve lightweight structure replacement of layers with poor pruning effect. On this basis, the model processed with structure replacement is re-trained to eventually build the final model. Lastly, the optimized YOLOv5n and YOLOv5s are tested respectively in single-object detection and multi-object detection scenarios. The test results show that the method proposed in this research is superior to the conventional lightweight method on ARM devices. In the character detection scenario, the size of the optimized YOLOv5n model is just 0.68 MB, and the detection speed can reach 45 fps when the single-core CPU is deployed on ARM devices, which can well meet the realtime requirements and also greatly reduce the difficulty and hardware cost of the deployment on side devices.