In order to solve the problem that it is difficult to identify the illegal use of mobile phones accurately and efficiently in some places with special regulations on mobile phone use, this paper proposes an improved SSD model to detect the illegal use of mobile phones. The SSD model is used to obtain the location and area classification of the primary target, and the improved DenseNet model is used to determine the primary target frame, so as to obtain the accurate mobile phone detection boundary box. In order to improve the data preprocessing process, the strategy of combining data amplification with image quality improvement is adopted. The experimental results on the mobile phone set built by ourselves verifying the effectiveness of these improved strategies. The location accuracy of the improved SSD model can reach 911%, and the recognition accuracy can reach 98%, which is 35% higher than the original SSD model. The improved SSD model has the characteristics of high recognition accuracy and high positioning accuracy, which can provide theoretical basis and technical support for intelligent recognition of illegal use of mobile phones.