Image fusion technology plays a vital role in the field of image analysis. In order to retain the details of the source images and improve the contrast of fusion image, a novel image fusion method is proposed based on visual weight map and multiscale decomposition. Firstly, bilateral filter with varied parameters is used for the multiscale decomposition of two source images. Then, visual weight map in each decomposition level is calculated and different weights are assigned to the different decomposition levels. Finally, fused image is generated by synthetizing these results. The image information is totally retained due to the decomposition without upsampling and downsampling. Moreover, this method can also overcome some of the wellknown problems in pixel level fusion such as blurring effects and high sensitivity to noise. Experimental results by using four metrics show that the proposed algorithm obtains dramatically improved performance compared to the other five stateoftheart algorithms. In addition, the computation time of our approach is less than 0.1 seconds, which is much better than other methods. The details and contrast of fused image are enhanced, and the computation time is reduced dramatically.