The traditional Weber local descriptor (WLD) algorithm has limitation in analyzing the center and neighboring pixels of the gray relationship. To identify facial expression accurately, a facial expression method based on Weber Gradient Coding(WGC) is proposed in this paper. First, the Weber Gradient encoding to the horizontal, vertical and diagonal gradient is respectively calculated to produce the differential excitation. Then, an optimized WGC operator based on horizontal and diagonal prior principle (WGC_HD) is proposed. Finally, the SVM classifier is used to implement the facial expression recognition based on row block WGC_HD feature. The experiments on the proposed method are performed using JAFFE and CohnKanade (CK), the average recognition rate is 95.49%, 9763% and the average duration of the extraction is 12.30 ms and 31.54 ms, respectively. The row block WGC_HD characteristics considering the difference of pixels of different gradient direction well expressed in local structure information of facial images and has lower time complexity. The recognition results of proposed method are better than those of the typical facial expression recognition method.