In order to evaluate the effect of different color transfer algorithms in different color space in the infrared and visible image fusion system, in YCbCr color space and lαβ color space, using objective evaluation method and subjective evaluation method, the global color transfer algorithm and pointtopoint color transfer algorithm are used to compare the effect of a set of images. Through the experimental comparison, the result of the objective evaluation method is consistent with the subjective evaluation method. It is concluded that, in the YCbCr space, the global color transfer effect is better than the pointtopoint color transfer effect, while in the lαβ space, the pointtopoint color transfer effect is better than the global color transfer effect.