An edge-preserving smoothing algorithm based on local Gaussian mean-difference variation is proposed to address the issue of detail not being preserved during the process of image smoothing. Firstly, a local Gaussian mean-difference variational operator is established by statistical analysis. To differentiate between structure and texture, the operator is employed to quantify the difference between the local gradient and the gradient after Gaussian filtering. Secondly, a local Gaussian mean-difference variational smoothing model is developed, and a sparse solution is used to produce the initial smooth image. Finally, an isolated noise removal model is suggested to address the issue of texture residue in images with complex texture. The model adjusts pixel values using an adaptive window and eliminates texture residue from the initial smooth image without changing the structure. It has been demonstrated through subjective and objective experiments that this algorithm produces smoothing results of superior quality than traditional algorithms. Evaluation indicators improved by 0.7% overall. Extended experiments verify the algorithm's applicability and efficiency enhancement potential across various visual tasks, including compression artifact removal, HDR tone mapping, image dehazing, and accelerated Laplacian pyramid.