In the field of medical image segmentation, noise interference from different devices or parameters negatively affects the generalization of the model. To solve the problem of segmentation model performance degradation caused by different noise sources, a two-stage efficient segmentation model NANet with adaptive noise pollution is proposed.The model mainly consists of two parts: denoising and segmentation. The denoising part uses an unsupervised U-shaped autoencoder structure combined with a frequency-domain learnable denoising module to mitigate the noise-induced target-domain differences. The segmentation part builds modules based on the DeepLabv3+architecture, focusing on improving the segmentation accuracy and generalization ability of targets in noise-polluted ultrasound images. Experiments were conducted to test the segmentation performance of the model on four different ultrasound datasets contaminated with uniform, Gaussian and pretzel noise. The experimental results show that the Dice coefficients of the proposed NANet for segmentation on the original image of the brachial plexus neural ultrasound dataset, the image data contaminated by uniform, Gaussian, and pretzel noises are 79.71%, 78.51%, 79.75%, and 79.1%, respectively. Comparison experiments show that the segmentation performance of NANet on ultrasound images with different noise types is significantly better than that of traditional segmentation methods, in which the accuracy is improved by more than 10% compared to U-Net, DeepLabv3+, and AttUNet methods on both Gaussian and pretzel noise images. The results validate the robustness and generalization ability of the proposed NANet in different noise environments.