A multimodal image fusion method based on the joint sparse model (JSM), multiscale dictionary learning, and a structural similarity index (SSIM) is presented. As an effective signal representation technique, JSM is derived from distributed compressed sensing and has been successfully employed in many image-processing applications such as image classification and fusion. The highly redundant single dictionary always has difficulty satisfying the correlations between images in traditional JSM-based image fusion. Therefore, the proposed fusion model learns a more compact multiscale dictionary to effectively combine the multiscale analysis used in nonsubsampled contourlet transformation with the single-scale joint sparse representation used in image domains to solve the issues of single-scale sparse fusion and to improve fusion quality. The experimental results demonstrate that the proposed fusion method obtains the state-of-the-art performances in terms of both subjective visual quality and objective metrics, especially when fusing multimodal images.
Infrared and visible images possess different types of simultaneous information, but there is a correlation between them. Traditional convolution sparse representation fusion considers the individual characteristics of each image but not the correlation between infrared and visible images. This results in insufficient detail retention and low contrast. To overcome these issues, joint convolution sparse coding is introduced, and a novel visible/infrared image fusion method is proposed. First, low-pass decomposition is used to decompose the source image into low- and high-pass components. Subsequently, joint convolutional sparse coding and a “choose-maximum” fusion strategy are used to fuse base layers, and the "absolute-maximum" is used for detail layers. Finally, image reconstruction is performed on the low and highpass components to obtain a final fused image. The proposed method not only avoids patch-based sparse fusion, which can destroy the image’s global structural features, but also fully integrates related information between infrared and visible images. Four groups of typical infrared and visible images are used for fusion experiments to verify the superiority of the proposed algorithm. The experimental results show that the proposed fusion algorithm provides optimal performance in subjective visual effects and objective evaluation indicators. Compared with the fusion method based on convolution sparse representation, three Q-series objective evaluation indicators increased by 3.83%, 5.31%, and 0.48%, respectively.
An adaptive joint sparsity model (JSM) is presented for multimodal image fusion. As a multisignal modeling technique, JSM, which is derived from distributed compressed sensing, has been successfully employed in multimodal image fusion. In traditional JSM-based fusion, a single dictionary learned by K-singular value decomposition (SVD) has higher coherence yet may result in potential visual confusion and misleading. In the proposed model, we first learn a plurality of subdictionaries and use a supervised classification approach based on gradient information. Then, one of the learned subdictionaries is adaptively applied to JSM to obtain the common and innovative sparse coefficients.. Finally, the fused image is reconstructed by the fused sparse coefficients and the adaptive dictionary. Infrared-visible images and medical images were selected to test the proposed approach. The results were compared with those of traditional methods, such as the multiscale transform-based methods, JSM-based method, and adaptive sparse representation (ASR) model-based method. Experimental results on multimodal images demonstrate that the proposed fusion method can obtain better performance than the conventional JSM-based method and ASR-based method in terms of both visual quality and objective assessment.