Due to the non convexity and combinatorial characteristics of heterogeneous networks , it is still very challenging to combine user association and resource allocation to achieve the optimal global strategy that maximizes both energy efficiency ( EE ) and spectral efficiency ( SE ) simultaneously . The method based on deep reinforcement learning (DRL ) has become a necessary solution for maximizing joint EE -SE performance while ensuring the quality of service (QoS ) of user equipments ( UEs ) in heterogeneous networks . In addition , to solve the problem of high computational complexity in the state action space, the double DQN algorithm with multi-agent dueling architecture ( MAD3QN) was introduced to obtain almost optimal control strategies. The simulation results show that the MAD3QN algorithm has increased system capacity by 9.2% and 18.2% respectively compared to the DDQN algorithm and DQN algorithm , and improved joint EE-SE performance by 8.5% and 16.6% respectively, enhancing the effectiveness of the system.