Convolutional operation only extracted local time-frequency information, and cannot effectively mine the relevant information between spectra. In order to solve this problem, a spectrum shift densenet was proposed. The module adopted structure of dense convolutional module, and the spectrum shift module was used to realize the information interaction between the spectra. It replaced the down-sampling operation between spectra and extracted the global feature from the spectrum. Meanwhile, it avoided the loss of information in the down-sampling process and further improved the quality of the spectrum feature maps. The proposed method was verified on two widely used dataset ESC10 and ESC50 respectively. The classification accuracy of ESC10 and ESC50 datasets is 96.00% and 88.75% respectively. Compared with the existing networks,the accuracy is improved by 2.1% and 2.25%. Comparde with convolutional neural networks based other methods, the proposed module can effectively mine more time-frequency information and has higher accuracy.