The accuracy of power dispatching speech recognition system is related to the effect of language model. In order to improve the accuracy of power dispatching speech recognition, this paper proposes a class label language model based on double dictionaries (general dictionary and power dispatching domain word dictionary). The model improves the n-gram language model and adds class label information, so as to improve the accuracy of power dispatching speech recognition. At the same time, a joint method of word segmentation and part of speech tagging based on double dictionaries is proposed. The system is used for word segmentation and class label labeling of corpus, and then improves the adaptability of class label language model based on double dictionary to power dispatching language. Finally, the comparison experiments between the proposed language model and the common statistical language models are carried out on the collected command set of power dispatching. In addition, the joint system and other word segmentation and part of speech tagging systems are compared by experiments. The simulation results show that the efficiency of word segmentation and part of speech tagging is higher in the joint system. Considering the comprehensive factors of semantic information, dictionaries, word segmentation and part of speech tagging system, the error rate of the proposed model in power dispatching language recognition is only 4.14%.