This paper studies the moving UAV to suppress the interference by dynamically sensing and learning the interference wave direction, and adjusting the beamforming strategy in real time. Aiming at the problem that the UAV can not obtain all the actions of the jammer for strategy training in the actual scene, a method of using the cooperation within the cluster to collect the action data of the jammer to supplement the training data is proposed to improve the anti-jamming of the cluster. The beamforming decision is modeled as a Markov decision process. Based on the deep reinforcement learning architecture, a data Aided Cooperative spatial anti-jamming algorithm for UAV cluster is proposed. The simulation results show that when the auxiliary data reaches 40%, 60% and 80%, the system throughput is improved by 33%, 55% and 70% respectively. It is verified that the method proposed in this paper can effectively improve the cooperative anti-jamming ability of UAV.