Fiber coating is very important to the performance of fiber. In the process of measuring the geometric parameters of the coating layer, because the cutting process of the coating layer is easy to produce a large deformation, and often causes the edge fuzzy, so the method of direct edge fitting to obtain the geometric size of the coating layer will cause a large error. In this paper, the wavelet threshold method is used to remove the noise part in the high-frequency signal of the image to protect the edge information. The effective edge is extracted by the Canny calculation of adaptive threshold. The variable radius is used to limit the edge in the case of large deformation, and the geometric parameters are obtained by function fitting. Taking the diameter of a coating layer as an example, the diameter measured by the standard instrument is 185.298µm, and the repeat measurement accuracy is 0.112µm. When the coating layer is not cut properly, the measurement value of the instrument becomes 184.516µm and the measurement accuracy becomes 0.198µm. The measured values before and after the method in this paper are 184.599µm and 184.613µm, and the measurement accuracy is 0.054µm and 0.075µm respectively. Experimental comparison shows that the method in this paper effectively improves the accuracy and stability of the measurement of the geometric parameters of the optical fiber coating.