Handwritten digit recognition has important application value in today′s society, and has broad application prospects in finance, social networking, education, communication and other fields. Handwritten digit recognition is a branch of optical character recognition technology. The common methods are identified by BP neural network, but there are three defects in BP neural network, such as local minimum, slow learning speed, and structure selection, and the optimization algorithm is used to optimize its structure. In this paper, genetic algorithm is used to optimize the initial threshold, initial weight and structure of BP neural network to overcome its shortcomings. The study of handwritten digital recognition as an object is carried out. The results of Matlab simulation show that the BP neural network optimized by genetic algorithm has the advantages of higher recognition accuracy, stronger generalization ability, faster convergence speed and stronger practicability, which provides a good theoretical basis for handwritten digital recognition.