The existing fusion methods cannot adjust fused images, adaptively according to the requirements of follow-up image processing steps, as well as the merits of different fusion methods, and are not easy to be integrated. Since a data assimilation system can integrate the advantage of a model operator and observer operator, a fusion framework based on the data assimilation concept is proposed, which can adaptively fuse different remote sensing images. Under this framework, a fusion method based on an independent component analysis and àtrous wavelet transform is used as a model operator, and another fusion method based on nonsubsampled contourlet transform is used as an observation operator. Meanwhile, image quantitative evaluation indicators are used as an objective function. Then, the genetic particle swarm algorithm is employed to optimize the objective function in order to gain a more suitable image. Finally, three sets of panchromatic images and multispectral images are used in experiments. The results show that the proposed algorithm can adjust fusion results adaptively, according to a particular objective function.