Traditional machine learning based methods use hand-crafted features in the inertial data to achieve the task of human activity recognition (HAR). However, as these features are normally without abstract high-level knowledge, the recognition rate is thus limited. Deep learning based methods, on the other hand, can avoid the aforementioned disadvantage by learning high-level features through labeled data. In this paper, the convolutional long short-term deep neural networks (CLDNN) is adopted for solving the HAR problem. This network has the features of both the convolutional neural network (CNN) and the recurrent neural network (RNN), which can extract features of different levels and can adopt the correlations in time sequences. Moreover, we use the GRU instead of LSTM as the gated cell of the RNN, which can make the network lighter. Through experiments adopting open source data, we can show that our method has 3% and 7% better recognition rate than CNN and RNN respectively, the training time and forward recognition time has decreased by 14% and 10% respectively, if replace the LSTM with GRU cell.