Most of the existing unsupervised domain adaptive fault diagnosis methods are only implemented based on a single domain signal, and the extracted fault information is not comprehensive enough. Only focus on realizing the edge distribution alignment of source and target domain features, ignoring the conditional distribution differences of samples, which limits the improvement of diagnostic accuracy. To overcome the above problems, a cross-domain fault diagnosis method of rolling bearings based on joint distribution offset differences (JDOD) is proposed. Two structurally consistent CNNs are used to extract the time-domain and frequency-domain features of the signal respectively to obtain more complete fault information. Joint distribution offset difference is proposed to realize edge distribution alignment and conditional distribution alignment of different domain features. Comparing experiments with various advanced methods on two multi-condition bearing datasets, the average diagnostic accuracy of more than 99% is obtained. The experimental results show that the joint distribution offset difference effectively improves the cross-domain fault diagnostic accuracy.