Due to the mutual interference of the ultrasonic scattering echoes in the imaging process, it may cause speckle noise in the formed medical ultrasonic images and it is difficult to distinguish from the human body structure, such as organ, tissue, etc, so that brining about complications to the later clinical diagnosis and image subsequent processing. In order to process the speckle noise in ultrasound images, a noise reduction model is proposed for adaptive averagefiltering ultrasound image based on support vector machine (SVM). The method uses the classification characteristics of SVM to distinguish the noise signal and the nonnoise signal in the ultrasound image, then combines the SVM classification result and the averagefiltering to denoise for the noise image. This operate can ensure the tissue area and detail characteristic of the medically noisy image are maximumly retain while the noise area is maximumly smooth. In the experimental part, the method used on the physical body membrane and human ultrasound liver image respectively. The results show that the proposed method can effectively suppress and reduce the speckle noise in the ultrasound image, and retain its edge features, and the signalnoise ratio of denoised image is increased. It can prove that the proposed method is useful for medical ultrasound image denoising.