Pansharpening via sparse regression

Songze Tang; Liang Xiao; Pengfei Liu; Lili Huang; Nan Zhou; Yang Xu

doi:10.1117/1.OE.56.9.093105

28 September 2017 Pansharpening via sparse regression

Songze Tang, Liang Xiao, Pengfei Liu, Lili Huang, Nan Zhou, Yang Xu

Author Affiliations +

Optical Engineering, Vol. 56, Issue 9, 093105 (September 2017). https://doi.org/10.1117/1.OE.56.9.093105

Abstract

Pansharpening is an effective way to enhance the spatial resolution of a multispectral (MS) image by fusing it with a provided panchromatic image. Instead of restricting the coding coefficients of low-resolution (LR) and high-resolution (HR) images to be equal, we propose a pansharpening approach via sparse regression in which the relationship between sparse coefficients of HR and LR MS images is modeled by ridge regression and elastic-net regression simultaneously learning the corresponding dictionaries. The compact dictionaries are learned based on the sampled patch pairs from the high- and low-resolution images, which can greatly characterize the structural information of the LR MS and HR MS images. Later, taking the complex relationship between the coding coefficients of LR MS and HR MS images into account, the ridge regression is used to characterize the relationship of intrapatches. The elastic-net regression is employed to describe the relationship of interpatches. Thus, the HR MS image can be almost identically reconstructed by multiplying the HR dictionary and the calculated sparse coefficient vector with the learned regression relationship. The simulated and real experimental results illustrate that the proposed method outperforms several well-known methods, both quantitatively and perceptually.

1. Introduction

Due to the launch cost and technical limitations, most Earth observation satellites, such as IKONOS, QuickBird, GeoEye, WorldView-2, GaoFen-1, and GaoFen-2, provided a high-resolution (HR) panchromatic (PAN) image and a low-resolution (LR) multispectral (MS) image with several spectral bands.¹^–⁴ With the increasing requirement of higher spatial and spectral resolution remote data for various applications, such as feature extraction, land-cover classification, and climate change evaluation, the ability to capture high-quality images with lower cost is becoming increasingly more important. Pansharpening is a special data fusion, which emerged as the practical requirement. It integrates the complementary spectral and spatial characteristics from the provided MS image and the PAN image into the desired product.² Up to now, a large number of pansharpening approaches have been developed. Broadly, three methodologies have been commonly used, namely, the component substitution, the high-frequency information injection, and the model-based methods.

The basic framework of component substitution methods is to transform the MS image into other space using a suitable transformation, and then the intensity channel or the first principal component is substituted by the PAN image. The classical component substitution methods are the intensity–hue–saturation (IHS)⁴ and principal component substitution.⁵ These methods may yield poor results in terms of spectral fidelity.⁶ In addition, due to an inadequate spectral model of the pansharpening method, spectral distortion is also generated. Some variant improvements to the original IHS-based method have been proposed. Rahmani et al.⁷ proposed an adaptive IHS (AIHS) method, which tried to represent the intensity component by an adaptive linear combination of the MS bands with the combination coefficients obtained by solving an optimization problem. Masoudi and Kabiri⁸ proposed a new IHS method using texture analysis and genetic algorithm adaption.

The high-frequency information injection methods can be briefly summarized as the sequential operations of details extraction and injection.⁹^–¹² Since most of the directional and structural information is contained in the PAN image, several researchers proposed using wavelets¹¹ and contourlet transforms¹² to capture it from the PAN image. Then, the details missed in the LR MS image can be extracted and supplemented from the PAN image. Compared with the component substitution methods, although this method preserved better spectral information, the spatial distortions may occur accompanied with some blurring and artifacts.

The model-based fusion approach is another important category. On the basis of the image restoration, some research regards the solution of the fused image as an inverse optimization problem.¹³^–¹⁷ Recently, inspired by sparse representation techniques, some researchers have achieved great success in data fusion.¹⁸^–²¹ The initial work is proposed by Li and Yang.¹⁸ Then, Jiang et al.¹⁹ proposed a two-step sparse coding method with patch normalization (PN-TSSC). Zhu and Bamler²⁰ proposed a new pansharpening method named sparse fusion of images (SparseFI), which explores the sparse representation of MS image patches in a dictionary trained only from the PAN image. To take into account the signal correlation among individual MS channels, the jointly SparseFI (J-SparseFI) algorithm was proposed.²¹ Although these methods perform well, the sparse coefficients of LR MS image patches are assumed to be identical to the corresponding HR MS patches. Due to complex and variant image content in the real-world images, the mappings between sparse coefficients of LR MS and that of HR MS should be complex.²²^,²³

To address this issue, we propose a pansharpening approach via sparse regression aimed at finding the intrinsic and implicit relationship among the sparse coefficients to improve the robustness and stability of the remote sensing image pansharpening. To achieve this goal, on the one hand, we learn a pair of compact dictionaries relied on the sampled patch pairs from the high- and low-resolution images. The learned dictionary pair can greatly characterize the structural information of the LR MS and HR MS images. On the other hand, taken the complex relationship between the coding coefficients of LR MS and HR MS images into account, we model the complex relationship by a ridge regression and an elastic-net regression. The ridge regression characterizes the relationship of intrapatches. The elastic-net regression describes the relationship of interpatches. The theoretical analyses and experimental results in this paper indicate that the proposed method can generate competitive fusion results. The flowchart of the proposed method is summarized in Fig. 1.

Fig. 1

The flowchart of the proposed method.

2. Pansharpening via Mapping of Sparse Coefficients

In this section, the scheme of dictionary learning and the ridge regression of intrapatches are introduced at first. Thereafter, the interpatches regression mapping based on elastic-net model is discussed in detail. Finally, we present the high-resolution MS images reconstruction.

2.1.

Dictionary Learning and the Ridge Regression of Intrapatches

The HR PAN image patches ${X_{P}^{(b)}}_{b = 1}^{B}$ and the corresponding LR PAN image patches ${Y_{P}^{(b)}}_{b = 1}^{B}$ have the sparse coefficients ${α_{H}^{(b)}}_{b = 1}^{B}$ and ${α_{L}^{(b)}}_{b = 1}^{B}$ under the corresponding dictionaries $D_{H}$ and $D_{L}$ , respectively. Inspired by Wang et al.,²² we make an assumption that there is an implicit mapping function $M \in R^{N \times N}$ between the sparse coefficients of LR MS and HR MS patch pairs. In addition, the sparse coefficients of LR PAN and HR PAN patch pairs share the mapping function (see Fig. 2), which can be modeled by the following linear ridge equation:

Eq. (1)

α_{H}^{(b)} = M α_{L}^{(b)} + ϵ,

where

ϵ

is an unknown error with zero mean. Always, the standard approach is known as ordinary least squares(OLS), which seeks to minimize the sum of square residuals. Mathematically, it solves Eq. (1) of the form

Eq. (2)

\underset{M}{\arg \min} \sum_{b = 1}^{B} {‖ α_{H}^{(b)} - M α_{L}^{(b)} ‖}_{2}^{2} .

Fig. 2

The relationship of the coefficients among the intrapatches.

However, OLS often does poorly in prediction. Thus, corresponding penalization techniques have been proposed to improve OLS to get a particular solution with desirable properties. For example, ridge regression²⁴ and lasso²⁵ are two popular and representative methods.

In our work, to get the mapping function $M$ with low computational burden and stable least square solution, we impose an $F$ -norm regularization penalty. The regularization term is included in the above-mentioned minimization

Eq. (3)

\underset{M}{\arg \min} \sum_{b = 1}^{B} {‖ α_{H}^{(b)} - M α_{L}^{(b)} ‖}_{2}^{2} + β {‖ M ‖}_{F}^{2} .

To enforce that the image patch pairs have the corresponding sparse coefficients with respect to HR dictionary $D_{H}$ and LR dictionary $D_{L}$ , a joint learning model below is proposed to find the desired dictionary pair as well as the desired intrapatches mapping

Eq. (4)

\underset{D_{H}, D_{L}, α_{H}, α_{L}, M}{\arg \min} \sum_{b = 1}^{B} {‖ X_{P}^{(b)} - D_{H} α_{H}^{(b)} ‖}_{2}^{2} + \sum_{b = 1}^{B} {‖ Y_{P}^{(b)} - D_{L} α_{L}^{(b)} ‖}_{2}^{2} + λ_{1} \sum_{b = 1}^{B} {‖ α_{H}^{(b)} ‖}_{1} + λ_{2} \sum_{b = 1}^{B} {‖ α_{L}^{(b)} ‖}_{1} + λ_{3} \sum_{b = 1}^{B} {‖ α_{H}^{(b)} - M α_{L}^{(b)} ‖}_{2}^{2} + λ_{4} {‖ M ‖}_{F}^{2} s.t. {‖ D_{H, i} ‖}_{2}^{2} \leq 1, {‖ D_{L, i} ‖}_{2}^{2} \leq 1, i = 1,2, \dots, N .

In the above equation, the PAN image and its degraded version are denoted by

X_{P}

and

Y_{p}

, respectively.

X_{P}^{(b)}

and

Y_{P}^{(b)}

are the

b

’th patch.

λ_{1}

,

λ_{2}

,

λ_{3}

, and

λ_{4}

are the regularized parameters, and the terms

D_{H, i}

and

D_{L, i}

are the atoms of the dictionaries

D_{H}

and

D_{L}

, respectively.

Given $X_{P} = [X_{P}^{(1)}, X_{P}^{(2)}, \dots, X_{P}^{(B)}]$ , $Y_{P} = [Y_{P}^{(1)}, Y_{P}^{(2)}, \dots, Y_{P}^{(B)}]$ , $α_{H} = [α_{H}^{(1)}, α_{H}^{(2)}, \dots, α_{H}^{(B)}]$ , $α_{L} = [α_{L}^{(1)}, α_{L}^{(2)}, \dots, α_{L}^{(B)}]$ , ${‖ α_{H} ‖}_{1,1} = \sum_{b = 1}^{B} {‖ α_{H}^{(b)} ‖}_{1}$ , and ${‖ α_{L} ‖}_{1,1} = \sum_{b = 1}^{B} {‖ α_{L}^{(b)} ‖}_{1}$ , Eq. (4) can be rewritten as follows:

Eq. (5)

\underset{D_{H}, D_{L}, α_{H}, α_{L}, M}{\arg \min} {‖ X_{P} - D_{H} α_{H} ‖}_{F}^{2} + {‖ Y_{P} - D_{L} α_{L} ‖}_{F}^{2} + λ_{1} {‖ α_{H} ‖}_{1,1} + λ_{2} {‖ α_{L} ‖}_{1,1} + λ_{3} {‖ α_{H} - M α_{L} ‖}_{F}^{2} + λ_{4} {‖ M ‖}_{F}^{2} s.t. {‖ D_{H, i} ‖}_{2}^{2} \leq 1, {‖ D_{L, i} ‖}_{2}^{2} \leq 1, i = 1,2, \dots, N .

Obviously, the above objective function is nonconvex in all of the variables but is convex in one of them with the other fixed.²⁶^,²⁷ Thus, initialized the mapping function

{M}

as an identity matrix and the dictionary pairs by DCT basis, the optimization performs in an alternative scheme over three stages. We call them “sparse coefficients update,” “dictionary update,” and “mapping function update,” which are corresponding to Eqs. (6)–(11). In other words, Eq. (5) is solved by translating to three subproblems.

In the sparse coefficients update stage, the mapping function $M$ and dictionary $D$ are fixed, and we can get the sparse coefficients $α_{H}$ and $α_{L}$ as follows:

Eq. (6)

\underset{α_{H}}{\arg \min} {‖ X_{P} - D_{H} α_{H} ‖}_{F}^{2} + λ_{1} {‖ α_{H} ‖}_{1,1} + λ_{3} {‖ α_{H} - M α_{L} ‖}_{F}^{2},

Eq. (7)

\underset{α_{L}}{\arg \min} {‖ Y_{P} - D_{L} α_{L} ‖}_{F}^{2} + λ_{2} {‖ α_{L} ‖}_{1,1} + λ_{3} {‖ α_{H} - M α_{L} ‖}_{F}^{2} .

Different from the traditional sparse coding, each equation has an extra

F

-norm regularization term. To simplify them, we combine the first and final terms and rewrite these equations as like the form of traditional sparse coding. Given

{\bar{X}}_{P} = [\begin{matrix} X_{P} \\ \sqrt{λ_{3}} M α_{L} \end{matrix}]

and

{\bar{D}}_{H} = [\begin{matrix} D_{H} \\ \sqrt{λ_{3}} I \end{matrix}]

, Eq. (6) has the following form

Eq. (8)

\underset{α_{H}}{\arg \min} {‖ {\bar{X}}_{P} - {\bar{D}}_{H} α_{H} ‖}_{F}^{2} + λ_{1} {‖ α_{H} ‖}_{1,1},

where

I

is an identity matrix. Then, Eq. (8) can be calculated by least angle regression algorithm.²⁸

Similar to the sparse coefficients update stage, with the sparse coefficients ${α_{H}, α_{L}}$ fixed, we update the dictionaries according to the following equations:

Eq. (9)

\underset{D_{H}}{\arg \min} {‖ X_{P} - D_{H} α_{H} ‖}_{F}^{2} s.t. {‖ D_{H, i} ‖}_{2}^{2} \leq 1, i = 1,2, \dots, N,

Eq. (10)

\underset{D_{L}}{\arg \min} {‖ Y_{P} - D_{L} α_{L} ‖}_{F}^{2} s.t. {‖ D_{L, i} ‖}_{2}^{2} \leq 1, i = 1,2, \dots, N .

Equations (9) and (10) are quadratic programmings, Lagrange dual technology²⁹ can be used to solve

{D_{H}, D_{L}}

.

At last, we update the mapping function $M$ fixed the dictionaries and sparse coefficients

Eq. (11)

M^{*} = \underset{M}{\arg \min} {‖ α_{H} - M α_{L} ‖}_{F}^{2} + β {‖ M ‖}_{F}^{2},

where

β = \frac{λ_{4}}{λ_{3}}

. Equation (11) has a closed-form solution

Eq. (12)

M^{*} = α_{H} α_{L}^{T} {(α_{L} α_{L}^{T} + β I)}^{- 1} .

2.2.

Learning Interpatch Regression Mapping Based on Elastic-Net Model

The above-mentioned regression mapping learning model just express the relationship of the sparse coefficients of image intrapatches. In Sec. 2.2, the relationship between sparse coefficients $α_{H}^{(b)}$ and all of the sparse coefficients ${α_{L}^{(b)}}_{b = 1}^{B}$ of LR patches is taken into account.

Let ${α_{L}^{(b)}}_{b = 1}^{B}$ be the predictors and $α_{H}^{(b)}$ be the response.³⁰ Then, the model between response and predictors can be assumed as

Eq. (13)

α_{H}^{(b)} \approx \sum_{p = 1}^{B} α_{L}^{(p)} w_{b}^{(p)}, b = 1,2, \dots B .

It can be shown in Fig. 3. The aim of this relationship is to seek the weight vectors

w_{b} = {(w_{b}^{(1)}, w_{b}^{(2)}, \dots, w_{b}^{(B)})}^{T}

.

Fig. 3

The relationship of the coefficients among the interpatches.

Given $A = [α_{L}^{(1)}, α_{L}^{(2)}, \dots, α_{L}^{(B)}]$ ;, the response is centered, and the predictors are standardized. Then, we reformulate Eq. (13) as

Eq. (14)

α_{H}^{(b)} = {Aw}_{b} + n, b = 1,2, \dots B .

Based on elastic-net regression model,³⁰ we proposed the following model:

Eq. (15)

{\hat{w}}_{b} = \underset{w_{b}}{\arg \min} {‖ α_{H}^{(b)} - {Aw}_{b} ‖}_{2}^{2} + γ_{1} {‖ w_{b} ‖}_{1} + γ_{2} {‖ w_{b} ‖}_{2}^{2},

where

γ_{1} > 0

and

γ_{2} > 0

are the regularized parameters.

Let us define $\bar{A} = {(1 + γ_{2})}^{- \frac{1}{2}} [\begin{matrix} A \\ \sqrt{γ_{2}} I \end{matrix}]$ , ${\bar{α}}_{H}^{(b)} = [\begin{matrix} α_{H}^{(b)} \\ 0 \end{matrix}]$ , $γ = \frac{γ_{1}}{\sqrt{1 + γ_{2}}}$ , and ${\bar{w}}_{b} = [\begin{matrix} w_{b} \\ 0 \end{matrix}]$ . Then, the elastic-net model can be described as a lasso-type problem

Eq. (16)

{\hat{w}}_{b} = \underset{{\bar{w}}_{b}}{\arg \min} {‖ {\bar{α}}_{H}^{(b)} - {\bar{A} \bar{w}}_{b} ‖}_{2}^{2} + γ {‖ {\bar{w}}_{b} ‖}_{1} .

The forward–backward operator splitting algorithm³¹ is employed to solve the problem

Eq. (17)

{\begin{cases} v_{(b)}^{(n)} = {\bar{w}}_{b}^{(n)} - μ {\bar{A}}^{T} ({\bar{A} \bar{w}}_{b}^{(n)} - α_{H}^{(b)}) \\ {\bar{w}}_{b}^{(n + 1)} = \underset{{\bar{w}}_{b}}{\arg \min} \frac{1}{μ} {‖ {\bar{w}}_{b} - v_{(b)}^{(n)} ‖}_{2}^{2} + γ {‖ {\bar{w}}_{b} ‖}_{1} \end{cases},

where

μ \in (0,2 / ‖ {\bar{A}}^{T} \bar{A} ‖)

. Thus,

{\bar{w}}_{b}^{(n + 1)}

can be given by the classical iterative soft threshold method³²

Eq. (18)

{\bar{w}}_{b}^{(n + 1)} = shrink (v_{(b)}^{(n)}, \frac{μ γ}{2}),

where

shrink (v, δ) = \max (‖ v ‖ - δ, 0) sgn (v)

. We get

Eq. (19)

w_{b} = \frac{1}{\sqrt{1 + γ_{2}}} {\hat{w}}_{b} .

2.3.

High-Resolution Multispectral Image Reconstruction

After learned the dictionary-pairs $D_{H}$ , $D_{L}$ , the intrapatch mapping function $M^{*}$ and interpatches mapping function $W^{*}$ from the HR PAN image patches and the degraded ones, we can reconstruct the HR MS image under the following assumptions:

1. The input LR MS images ${Y_{MS, k}}_{k = 1}^{K}$ and LR PAN image share the LR dictionary $D_{L}$ .
2. The reconstructed HR MS images ${X_{MS, k}}_{k = 1}^{K}$ and HR PAN image share the HR dictionary $D_{H}$ .
3. The sparse coefficients of LR MS images $α_{LMS}$ and HR MS images $α_{HMS}$ share the learned mapping function $M^{*}$ and $W^{*}$ .

The sparse coefficients of the $k$ ’th band of the LR MS image ${Y_{MS, k}}_{k = 1}^{K}$ can be calculated by following step (each band is processed independently):

Eq. (20)

\underset{α_{LMS, k}}{\arg \min} {‖ Y_{MS, k} - D_{L} α_{LMS, k} ‖}_{F}^{2} + λ_{2} {‖ α_{LMS, k} ‖}_{1,1} .

Then, we generate the corresponding coding coefficients associate $α_{LMS, k}$ with the regression function $M^{*}$ and $W^{*}$ , respectively

Eq. (21)

{\bar{α}}_{HMS, k} = M^{*} α_{LMS, k},

Eq. (22)

{\hat{α}}_{HMS, k} = α_{LMS, k} W^{*} .

The sparse coefficients of the

k

’th band of the HR MS image

{X_{MS, k}}_{k = 1}^{K}

is

α_{HMS, k} = (1 - p) \cdot {\bar{α}}_{HMS, k} + p \cdot {\hat{α}}_{HMS, k}

, where

p

is the weight parameter (here,

p = 0.45

).

Finally, each band can be reconstructed as

Eq. (23)

X_{MS, k} = D_{H} α_{HMS, k} .

3. Experimental Results and Analysis

To assess the performance of the proposed method, both simulated experiments and real data experiments are carried out. The simulation experiments are based on the strategy proposed by Wald et al.³³ First, the original PAN and MS images are blurred with a low-pass filter and downsampled by a decimation factor 4 to obtain a degraded PAN image and MS images. Then, these degraded PAN and MS images are used to yield the HR MS images with the same spatial resolution to the original MS images. Finally, the fused HR MS images are compared with the original MS images. The QuickBird and WorldView-2 data are employed to test the performance of the proposed method. Five typical evaluation metrics are adopted to quantitatively evaluate the pansharpened results. The correlation coefficient (CC)³⁴ and root-mean-square error (RMSE) are calculated for each band between the fused MS images and the reference original MS image. Erreur relative globale adimensionnelle de synthèse (ERGAS)³³ and $Q_{4}$ ,³⁵ which are two comprehensive evaluation indexes, provide unique measures of the fusion performance for all the MS bands. Furthermore, the spectral angle mapper (SAM) index³⁴ is also considered to measure the spectral distortion. Smaller values of RMSE, SAM, and ERGAS tend to be achieved by a better fusion result, as do larger CC, and Q4 values. In the real data experiments, since there is no reference HR MS image, the “quality with no reference” (QNR) measurement is used to evaluate different pansharpened results objectively.³⁵

The proposed method is compared with five popular fusion algorithms: AIHS,⁷ Wavelet,¹¹ PN-TSSC,¹⁹ SparseFI,²⁰ and J-SparseFI.²¹ The results of AIHS method are gotten from the software developed by Rahmani et.al. The implementation of the AIHS method is available online in Ref. 36. The default parameters given in their implementations are adopted. Two levels of decompositions are used for the Wavelet method. The LR patch size in the SparseFI method is $7 \times 7$ . A total of 10,000 patch pairs are selected to construct the dictionary pairs. As to J-SparseFI method, patch size is $7 \times 7$ , and the dictionary size is 1000.

For our proposed pansharpening method, there are several parameters to be selected. In our experiments, we set the weight parameter $p = 0.45$ , the regularization parameters $λ_{1} = 0.01$ , $λ_{2} = 0.01$ , $λ_{3} = 0.1$ , $λ_{4} = 0.1$ and $γ_{1} = 0.3$ , $γ_{2} = 0.5$ , the patch size $5 \times 5$ , and the dictionary size 512. In Sec. 3.1, we provided a recipe about how to select these parameters to achieve a promising pansharpened result. Experimental results using parameters selected according to this recipe will be presented through the evolution curves and final fused images on different datasets.

3.1.

Parameter Analysis

In this section, we investigate the effect of the weight parameter $p$ and the regularization parameters $λ_{1}$ , $λ_{2}$ , $λ_{3}$ , $λ_{4}$ and $γ_{1}$ , $γ_{2}$ of the proposed method.

1. The effect of the weight parameter $p$ : The pansharpened performance obtained by the proposed method has strong links with the regularization parameter $p$ . To demonstrate it, we present the influence of the proposed method by choosing different weight parameters ( $p$ ). CC, SAM, and Q4 are evaluated to determine $p$ . The corresponding curved lines are plotted in Figs. 4 and 5, respectively. As shown in Fig. 4, when $p = 0$ , which can be regarded as the case of ridge regression of intrapatches, the performance of the proposed method is restricted. With the increase of $p$ , more benefits on performance can be gained. This implies that the interpatches regression model is also essential for the fused result. However, we should also see that the value could not be set too high. Therefore, with a proper regularization parameter $p$ , the proposed method will gain the best results. A similar conclusion can be drawn for the QuickBird data in Fig. 5. Among the above resulted images, quantitatively, the best pansharpening result is yielded at $p = 0.45$ .
2. The influence of the regularization parameters $λ_{1}$ , $λ_{2}$ , $λ_{3}$ , $λ_{4}$ and $γ_{1}$ , $γ_{2}$ : In this section, we probe how the regularization parameters affect the pansharpened performance of the proposed method. To test the influence, we tune one of them with the other fixed. Tables 1 and 2 are the objective assessment results of the proposed method on WorldView-2 data according to different values of $λ_{1}$ and $λ_{2}$ . Table 3 reports the influence of $γ_{1}$ on QuickBird data. But it should reinforce the point that the pansharpened results are consistent with different test datasets under different parameters. From the tables, we can see that the regularization parameters play important roles in the final pansharpened performance. Little worse performance will be obtained when small deviations of the regularization parameters are around the selected values. If one tunes them far away from the value set by us, some worse performance will appear in the final fused image. When $λ_{1}$ , $λ_{2}$ , and $γ_{1}$ are in the order of 0.01, 0.01, and 0.3, respectively, the proposed method can continually obtain a stable and optimal performance. The same thing happened again to other regularization parameters $λ_{3}$ , $λ_{4}$ , and $γ_{2}$ . For balancing the reconstruction error, we set $λ_{1} = 0.01$ , $λ_{2} = 0.01$ , $λ_{3} = 0.1$ , $λ_{4} = 0.1$ , $γ_{1} = 0.3$ , and $γ_{2} = 0.5$ .

Fig. 4

Performance comparison of the WorldView-2 data with different weight parameters.

Fig. 5

Performance comparison of the QuickBird data with different weight parameters.

Table 1

The effects of the proposed method under different λ1 on WorldView-2 data.

	λ1
	0.008	0.009	0.01	0.011	0.012	0.015	0.02	0.05	0.1	0.2	0.5
CC	0.9828	0.9829	0.9830	0.9830	0.9825	0.9826	0.9798	0.9821	0.9229	0.8511	0.7392
RMSE	0.0456	0.0452	0.0451	0.0455	0.0511	0.0498	0.0502	0.0510	0.7829	0.9525	1.4834
SAM	3.1892	3.1877	3.1816	3.1782	3.1940	3.2352	3.2355	3.2408	4.5768	6.2065	8.8587
ERGAS	3.5991	3.5708	3.5547	3.5912	3.6576	3.6766	3.6700	3.6701	4.2308	6.9293	9.0720
Q4	0.8195	0.8196	0.8197	0.8189	0.8192	0.8190	0.8191	0.8180	0.7877	0.7104	0.5266

The bold and italic values represent the best results.

Table 2

The effects of the proposed method under different λ2 on WorldView-2 data.

	λ2
	0.008	0.009	0.01	0.011	0.012	0.015	0.02	0.05	0.1	0.2	0.5
CC	0.9825	0.9828	0.9830	0.9829	0.9828	0.9820	0.9808	0.9779	0.9378	0.8821	0.7901
RMSE	0.0457	0.0453	0.0451	0.0448	0.0455	0.0498	0.0673	0.1998	1.0273	1.6532	2.7903
SAM	3.2211	3.1945	3.1816	3.1808	3.1949	3.5775	3.8128	4.0106	6.0818	6.9501	8.0002
ERGAS	3.5987	3.6213	3.5547	3.6111	3.5654	3.6799	3.7991	4.2257	5.9802	6.7733	8.7616
Q4	0.8195	0.8197	0.8197	0.8196	0.8194	0.8193	0.8193	0.8192	0.8050	0.7608	0.6191

The bold and italic values represent the best results.

Table 3

The effects of the proposed method under different γ1 on QuickBird data.

	γ1
	0.25	0.27	0.28	0.29	0.3	0.31	0.32	0.33	0.35	0.4	0.5
CC	0.9626	0.9630	0.9633	0.9632	0.9632	0.9632	0.9629	0.9625	0.9618	0.9587	0.9427
RMSE	0.0677	0.0661	0.0656	0.0652	0.0651	0.0653	0.0659	0.0673	0.0722	0.0865	0.0991
SAM	4.0602	4.0584	4.0571	4.0563	4.0563	4.0563	4.0576	4.0583	4.0626	4.0744	4.1378
ERGAS	5.3766	5.3689	5.3607	5.3595	5.3587	5.3592	5.3611	5.3639	5.3705	5.3876	5.4305
Q4	0.7850	0.7857	0.7861	0.7864	0.7865	0.7865	0.7862	0.7860	0.7852	0.7831	0.7804

The bold values represent the best results.

3.2.

Effects of Patch Size

As described in Sec. 2, our proposed method is based on patches learning. Thus, the effects of patch size on the pansharpening performance are evaluated in this section. To be fair, the dictionary size is fixed as 512. Four different patch sizes for LR PAN image are studied, including $3 \times 3$ , $5 \times 5$ , $7 \times 7$ , and $9 \times 9$ . Figure 6 shows how the patch size affects the visual quality of the fused results on WorldView-2 data. We can observe that the difference of spectral distortion is very small under different patch sizes. To illustrate well the performance change with the patches sizes, the quality indexes are calculated, where the average CC and RMSE of eight bands are presented. In addition, all the values of indexes are normalized to the range [0, 1]. The normalized results with respect to the different patch size are plotted in Fig. 7, where the horizontal axis is the patch size, and the vertical axis is the normalized results. Larger CC and Q4 indicate better fused result, and smaller RMSE, SAM, and ERGAS mean better result. Based on the curves in Fig. 7, it can be seen that when the patch size is $7 \times 7$ , the best performance of the proposed method is obtained. The performance of patch size $5 \times 5$ is slightly worse. However, the proposed method has less space complexity at this time. Taken into account a trade-off between the practical application in the future and the performance, we choose the patch size $5 \times 5$ in the following experiments.

Fig. 6

The effects of patch size on the pansharpening of WorldView-2 image: (a) $3 \times 3$ , (b) $5 \times 5$ , (c) $7 \times 7$ , (d) $9 \times 9$ , and (e) original HR MS image.

Fig. 7

Performance of the proposed method with different patch sizes.

3.3.

Effects of Dictionary Size

The above experimental results, we fix the dictionary size to be 512. In general, larger dictionaries should possess more expressive power and thus may yield more accurate approximation while increasing the computation cost. In this section, we evaluate the effect of dictionary size on pansharpening, i.e., the number of atoms in dictionary. From the sampled image patch pairs, we train four dictionaries of size 256, 512, 1024, and 2048 and apply them to the same remote sensing image. The results are evaluated both visually and quantitatively in Figs. 8 Fig. 9–10 and Table 4.

Fig. 8

The effects of dictionary size on the pansharpening of QuickBird image: dictionary size (a) 256, (b) 512, (c) 1024, and (d) 2048, and (e) the original HR MS image.

Fig. 9

The difference image of different dictionary sizes on the pansharpening of QuickBird image: dictionary size (a) 256, (b) 512, (c) 1024, and (d) 2048.

Fig. 10

The computational time on QuickBird image with dictionaries of different sizes (in seconds).

Table 4

The objective indexes of the pansharpened images with dictionaries of different sizes.

Dictionary size	CC	RMSE	SAM	ERGAS	Q4
256	0.9641	0.0815	4.1271	5.3470	0.7788
512	0.9642	0.0801	4.1023	5.2598	0.7820
1024	0.9638	0.0786	4.0882	5.1753	0.7852
2048	0.9640	0.0758	3.9789	4.9666	0.7932

The bold values represent the best results.

Figure 8 shows the fused results for the QuickBird image using dictionaries of different sizes. Human visual system is not sensitive to the weak spectral distortions. Always we justify the distortion from the color change.¹⁰ Thus, in Fig. 9, we display the difference of pixel values measured between each pansharpened image and the reference HR MS image. Deep blue represents the smallest difference, whereas the red means the largest difference. Figure 9 shows the difference image under different dictionary sizes. While there are not many visual differences for the results using different dictionary sizes from 256 to 2048, we indeed observe the artifacts will gradually diminish with larger dictionaries (i.e., the subtle differences in the yellow circle in Fig. 9). In Table 4, we list five indexes of the pansharpened image for dictionaries of different sizes. As shown in the table, using larger dictionaries will yield better quantitative indexes. However, the computation is approximately linear to the size of the dictionary, i.e., larger dictionaries will result in heavier computation. Figure 10 shows the computation time in seconds with the fused image. In practice, one chooses an appropriate dictionary size as a trade-off between pansharpening quality and computation cost. We find that dictionary size 512 can yield decent outputs while allowing fast computation.

3.4.

Simulation Results and Analysis

The size of the LR MS image in the simulated WorldView-2 data experiment is $128 \times 128 pixels$ with eight bands and the corresponding PAN image sized $512 \times 512 pixels$ . In Fig. 11(i), there are many varieties of ground objects, such as vegetation, buildings, and roads. In the aspect of the visual effects, the Wavelet method suffers from both spectral distortion and blocky artifacts in the building regions, as shown in Fig. 11(d). Meanwhile, the proposed method and J-SparseFI method show especially significant improvement in the spatial resolution. The details in the final fused images are as clear as that in the PAN image. AIHS and PN-TSSC methods are not good enough at improving the spatial resolutions. In Figs. 11(c) and 11(e), a lot of details are lost in the buildings, and the noise is introduced into the fused images. Compared with AIHS and PN-TSSC methods, SparseFI method is good in injecting more details. However, there are phenomena of the spectral distortions. The color of the whole fused images is dark. The visual effects are not close to the reference HR MS image.

Fig. 11

Pansharpening results obtained by different methods on WorldView-2 data: (a) LR MS image, (b) PAN, (c) AIHS, (d) Wavelet, (e) PN-TSSC, (f) SparseFI, (g) J-SparseFI, (h) proposed, and (j) HR MS image.

To evaluate the performance of various methods objectively, Table 5 presents the objective evaluations about different methods. On the whole, the proposed method demonstrates the best objective performance, i.e., ranking as the first for the CC, RMSE, SAM, ERGAS, and Q4. The CC value of J-SparseFI method is inferior to the proposed method. It means J-SparseFI method cannot preserve the spectral information as well as the latter. The high Q4 value represents its effectiveness of improvement of the spatial resolution. The low SAM and ERGAS values of AIHS, PN-TSSC, and SparseFI methods show their shortage of spectral information of preservation. We also noticed the values of Q4 are very high, which reflect that the SparseFI and J-SparseFI method have the advantage of improving the spatial resolution.

Table 5

Objective performance for different pansharpening methods on WorldView-2 data.

	AIHS	Wavelet	PN-TSSC	SparseFI	J-SparseFI	Proposed
CC	0.9712	0.9745	0.9779	0.9785	0.9816	0.9830
RMSE	0.0691	0.0911	0.0662	0.0594	0.0490	0.0451
SAM	5.3163	7.0537	4.0698	3.1437	3.0297	3.0009
ERGAS	5.4726	7.5020	4.1187	3.6593	3.6208	3.5547
Q4	0.7405	0.6712	0.8089	0.8172	0.8196	0.8197

Figure 12 shows the difference of pixel values measured between each pansharpened image and the reference HR MS image. We can see that all the fused images by different methods have a lot of areas with blue and deep blue from the corresponding difference images. Fortunately, the proposed method shows the best performance. The least area of the red part in Fig. 12(f) indicates that the proposed method has a small number of outliers, which is consistent with the above objective and subjective analysis.

Fig. 12

Difference image between each HR MS image and the reference high resolution MS image (deep blue means small differences and red means large differences). (a) AIHS; (b) Wavelet; (c) PN-TSSC; (d) SparseFI; (e) J-SparseFI; (f) Proposed.

3.5.

Experiments on Real Data

To verify the effectiveness of the proposed method in practical applications, we move on to conduct the experiments on the real data. Figures 13 and 14 present the fused results using six different methods. In Fig. 13, we can see that the AIHS method has the problem of textures overenhancement, as shown in the mountain. Figures 13(e)–13(g) are the pansharpened images provided by the PN-TSSC, SparseFI, and J-SparseFI, respectively, which can obtain promising results without causing obvious spectral and spatial distortion. Since our work illustrates the complex mapping relationship between the LR MS and HR MS image, experiments are also performed on complex urban scenes, as shown in Fig. 14. The final pansharpened results of various methods are presented in Fig. 14(c)–14(h). By the comparisons of all results, the component substitution method (AIHS) and the model-based methods (PN-TSSC, SparseFI, J-SparseFI, and proposed) outperform the high-frequency information injection method (Wavelet) in preserving the spectral information. Specifically, the proposed method produces the natural and satisfactory pansharpened images, which has similar spatial structures with the PAN image and similar spectral information with the MS image, as shown in Fig. 14(h). Table 6 and 7 show the quantitative assessment results.³⁵ The ranking of QNR scores can confirm that the proposed method is better than the other methods in sharpening the real data.

Fig. 13

Real data experimental results by different methods: (a) real MS image at 2-m spatial resolution, (b) real PAN image at 0.5-m spatial resolution, (c) AIHS, (d) Wavelet, (e) PN-TSSC, (f) SparseFI, (g) J-SparseFI, and (h) proposed method.

Fig. 14

Real data experimental results by different methods: (a) real MS image at 1.65-m spatial resolution, (b) real PAN image at 0.41-m spatial resolution, (c) AIHS, (d) Wavelet, (e) PN-TSSC, (f) SparseFI, (g) J-SparseFI, and (h) proposed method.

Table 6

Comparison of the proposed method with other methods on real data shown in Fig. 13.

	AIHS	Wavelet	PN-TSSC	SparseFI	J-SparseFI	Proposed
$D_{λ}$	0.0524	0.0376	0.0465	0.0623	0.0596	0.0352
$D_{s}$	0.1315	0.1003	0.0917	0.0667	0.0632	0.0618
QNR	0.8230	0.8659	0.8661	0.8752	0.8810	0.9052

Table 7

Comparison of the proposed method with other methods on real data shown in Fig. 14.

	AIHS	Wavelet	PN-TSSC	SparseFI	J-SparseFI	Proposed
$D_{λ}$	0.0837	0.1212	0.0706	0.0705	0.0673	0.0611
$D_{s}$	0.1761	0.1953	0.1418	0.1469	0.1407	0.1386
QNR	0.7549	0.7072	0.7976	0.7930	0.8015	0.8088

3.6.

Time Consumption

All the methods are implemented in MATLAB^® 2010b and run on an Intel Core i7@3.6-GHz PC with 32-GB RAM. For fusing a $512 \times 512$ PAN image and $128 \times 128 \times 4$ MS images, the AIHS and Wavelet methods need less than 1 s. The PN-TSSC, SparseFI, and J-SparseFI methods take about 147 s, 115 s, and 5 min, respectively. The running time of our proposed method is 133 s. Compared with these methods, the running time of the patch-based methods is still room for improvement. However, it is also reasonable to believe that with the rapid development in computer hardware and computation techniques, the time cost of the proposed method will soon no longer be an issue. We also can learn numerous projection matrices from the LR MS feature spaces to the corresponding HR MS feature spaces, which may decrease the computational time in the pansharpening phase. This will be investigated in our future work.

4. Conclusion

In this paper, combining ridge regression and elastic net of sparse representation, a pansharpening method is proposed for merging a PAN image with an MS image. The method used HR PAN image patches and their degraded ones to construct a training database. Then, the semicoupled dictionary learning method is used to train the LR–HR dictionary pair, which is to depict the MS structural information. The relationship between coding coefficients of LR MS and HR MS image patches is predicted by the within-patch ridge regression and among-patch elastic-net model. Our method can enhance the spatial resolution of MS images while reducing the distortion of spectral. The experimental results demonstrate that the proposed method can be compared with other state-of-the-art fusion methods.

Acknowledgments

This work was supported in part by the Fundamental Research Funds for the Central Universities (Grant No. LGZD201702), the National Natural Science Foundation of China (Grant Nos. 61302178, 61171165, 11431015, and 61571230), the Foundation of Guangxi Province (Grant No. 2014GXNSFAA118360), the Foundation of Jiangsu Province (Grant No. BK20161055), and the National Scientific Equipment Developing Project of China under Grant No. 2012YQ050250.

References

1.

T. M. Tu et al., “An adjustable pan-sharpening approach for IKONOS/QuickBird/GeoEye-1/WorldView-2 imagery,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 5 (1), 125 –134 (2012). http://dx.doi.org/10.1109/JSTARS.2011.2181827 Google Scholar

2.

H. T. Yin and S. T. Li, “Pansharpening with multiscale normalized nonlocal means filter: a two-step approach,” IEEE Trans. Geosci. Remote Sens., 53 (10), 5734 –5745 (2015). http://dx.doi.org/10.1109/TGRS.2015.2429691 IGRSD2 0196-2892 Google Scholar

3.

S. Z. Tang et al., “Pan-sharpening using 2D CCA,” Remote Sens. Lett., 6 (5), 341 –350 (2015). http://dx.doi.org/10.1080/2150704X.2015.1034882 Google Scholar

4.

T. M. Tu et al., “A fast intensity hue-saturation fusion technique with spectral adjustment for IKONOS imagery,” IEEE Geosci. Remote Sens. Lett., 1 (4), 309 –312 (2004). http://dx.doi.org/10.1109/LGRS.2004.834804 Google Scholar

5.

P. S. Chavez, S. C. Sides and J. A. Anderson, “Comparison of three different methods to merge multiresolution and multispectral data Landsat TM and SPOT panchromatic,” Photogramm. Eng. Remote Sens., 57 (3), 295 –303 (1991). http://dx.doi.org/10.1163/157006707X222768 Google Scholar

6.

Y. Zhang, “Understanding image fusion,” Photogramm. Eng. Remote Sens., 70 (6), 657 –661 (2004). http://dx.doi.org/10.1080/01431160410001662220 Google Scholar

7.

S. Rahmani et al., “An adaptive IHS pan-sharpening method,” IEEE Geosci. Remote Sens. Lett., 7 (4), 746 –750 (2010). http://dx.doi.org/10.1109/LGRS.2010.2046715 Google Scholar

8.

R. Masoudi and P. Kabiri, “New intensity-hue-saturation pan-sharpening method based on texture analysis and genetic algorithm-adaption,” J. Appl. Remote Sens., 8 (1), 083640 (2014). http://dx.doi.org/10.1117/1.JRS.8.083640 Google Scholar

9.

M. C. El-Mezouar et al., “A pan-sharpening based on the non-subsampled contourlet transform: application to WorldView-2 imagery,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 7 (5), 1806 –1815 (2014). http://dx.doi.org/10.1109/JSTARS.2014.2306332 Google Scholar

10.

X. Kang, S. Li and J. A. Benediktsson, “Pansharpening with matting model,” IEEE Trans. Geosci. Remote Sens., 52 (8), 5088 –5099 (2014). http://dx.doi.org/10.1109/TGRS.2013.2286827 IGRSD2 0196-2892 Google Scholar

11.

X. Otazu et al., “Introduction of sensor spectral response into image fusion methods: application to wavelet-based methods,” IEEE Trans. Geosci. Remote Sens., 43 (10), 2376 –2385 (2005). http://dx.doi.org/10.1109/TGRS.2005.856106 IGRSD2 0196-2892 Google Scholar

12.

A. G. Mahyari and M. Yazdi, “Panchromatic and multispectral image fusion based on maximization of both spectral and spatial similarities,” IEEE Trans. Geosci. Remote Sens., 49 (6), 1976 –1985 (2011). http://dx.doi.org/10.1109/TGRS.2010.2103944 IGRSD2 0196-2892 Google Scholar

13.

H. Shen, X. Meng and L. Zhang, “An integrated framework for the spatio-temporal-spectral fusion of remote sensing images,” IEEE Trans. Geosci. Remote Sens., 54 (12), 7135 –7148 (2016). http://dx.doi.org/10.1109/TGRS.2016.2596290 IGRSD2 0196-2892 Google Scholar

14.

R. C. Hardie, M. T. Eismann and G. L. Wilson, “MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor,” IEEE Trans. Image Process., 13 (9), 1174 –1184 (2004). http://dx.doi.org/10.1109/TIP.2004.829779 IIPRE4 1057-7149 Google Scholar

15.

R. Molina et al., “Variational posterior distribution approximation Bayesian super resolution reconstruction of multispectral images,” Appl. Comput. Harmon. Anal., 24 (2), 251 –267 (2008). http://dx.doi.org/10.1016/j.acha.2007.03.006 ACOHE9 1063-5203 Google Scholar

16.

L. Zhang et al., “Adjustable model-based fusion method for multispectral and panchromatic images,” IEEE Trans. Syst. Man Cybern. B, 42 (6), 1693 –1704 (2012). http://dx.doi.org/10.1109/TSMCB.2012.2198810 Google Scholar

17.

P. Liu et al., “Spatial-Hessian-feature-guided variational model for pan-sharpening,” IEEE Trans. Geosci. Remote Sens., 54 (4), 2235 –2253 (2016). http://dx.doi.org/10.1109/TGRS.2015.2497966 IGRSD2 0196-2892 Google Scholar

18.

S. Li and B. Yang, “A new pan-sharpening method using a compressed sensing technique,” IEEE Trans. Geosci. Remote Sens., 49 (2), 738 –746 (2011). http://dx.doi.org/10.1109/TGRS.2010.2067219 IGRSD2 0196-2892 Google Scholar

19.

C. Jiang et al., “Two-step sparse coding for the pan-sharpening of remote sensing images,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 7 (5), 1792 –1805 (2014). http://dx.doi.org/10.1109/JSTARS.2013.2283236 Google Scholar

20.

X. Zhu and R. Bamler, “A sparse image fusion algorithm with application to pan-sharpening,” IEEE Trans. Geosci. Remote Sens., 51 (5), 2827 –2836 (2013). http://dx.doi.org/10.1109/TGRS.2012.2213604 IGRSD2 0196-2892 Google Scholar

21.

X. Zhu, C. Grohnfeldt and R. Bamler, “Exploiting joint sparsity for pansharpening: the J-SparseFI algorithm,” IEEE Trans. Geosci. Remote Sens., 54 (5), 2664 –2681 (2016). http://dx.doi.org/10.1109/TGRS.2015.2504261 IGRSD2 0196-2892 Google Scholar

22.

S. Wang et al., “Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis,” in IEEE Conf. on Computer Vision and Pattern Recognition (CVPR ’12), 2216 –2223 (2013). http://dx.doi.org/10.1109/CVPR.2012.6247930 Google Scholar

23.

S. Tang et al., “Joint dictionary learning with ridge regression for pansharpening,” in IEEE Int. Geoscience and Remote Sensing Symp., 613 –616 (2015). http://dx.doi.org/10.1109/IGARSS.2015.7325838 Google Scholar

24.

A. Hoerl and R. Kennard, “Ridge regression,” Encyclopedia of Statistical Sciences, 129 –136 Wiley, New York (1988). Google Scholar

25.

R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. R. Stat. Soc. B, 58 (1), 267 –288 (1996). JSTBAJ 0035-9246 Google Scholar

26.

H. Zhang et al., “Close the loop: joint blind image restoration and recognition with a sparse representation prior,” in Proc. IEEE Int. Conf. on Computer Vision, 770 –777 (2011). http://dx.doi.org/10.1109/ICCV.2011.6126315 Google Scholar

27.

S. Tang et al., “Edge and color preserving single image superresolution,” J. Electron. Imaging, 23 (3), 033002 (2014). http://dx.doi.org/10.1117/1.JEI.23.3.033002 JEIME5 1017-9909 Google Scholar

28.

B. Efron et al., “Least angle regression,” Ann. Stat., 32 (2), 407 –499 (2004). http://dx.doi.org/10.1214/009053604000000067 Google Scholar

29.

H. Lee et al., “Efficient sparse coding algorithms,” in Advances in Neural Information Processing Systems, (2006). Google Scholar

30.

H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” J. R. Stat. Soc. B, 67 (2), 301 –320 (2005). http://dx.doi.org/10.1111/rssb.2005.67.issue-2 Google Scholar

31.

P. L. Combettes and V. R. Wajs, “Signal processing by proximal forward-backward splitting,” Multiscale Model. Simul., 4 (4), 1168 –1200 (2006). http://dx.doi.org/10.1137/050626090 Google Scholar

32.

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imaging Sci., 2 (1), 183 –202 (2009). http://dx.doi.org/10.1137/080716542 Google Scholar

33.

L. Wald, T. Ranchin and M. Mangolini, “Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images,” Photogramm. Eng. Remote Sens., 63 (6), 691 –699 (1997). http://dx.doi.org/10.1016/S0924-2716(97)00008-7 Google Scholar

34.

L. Alparone et al., “Comparison of pansharpening algorithms: outcome of the 2006 GRSS data fusion contest,” IEEE Trans. Geosci. Remote Sens., 45 (10), 3012 –3021 (2007). http://dx.doi.org/10.1109/TGRS.2007.904923 IGRSD2 0196-2892 Google Scholar

35.

L. Alparone et al., “A global quality measurement of pan-sharpened multispectral imagery,” IEEE Geosci. Remote Sens. Lett., 1 (4), 313 –317 (2004). http://dx.doi.org/10.1109/LGRS.2004.836784 Google Scholar

36.

S. Rahmani, M. Strait and D. Merkurjev, “Pansharpening GUI,” (2008) http://www.math.ucla.edu/~wittman/pansharpening/ Google Scholar

Biography

Songze Tang is currently an assistant professor in the Department of Criminal Science and Technology, Nanjing Forest Police College. He received his BS degree in information and computation science from Anhui Agriculture University, Hefei, China, in 2009, and his PhD in computer science and technology from Nanjing University of Science and Technology, Nanjing, in 2015. His research interests include computer vision and biometrics. He serves as a reviewer for three international academic journals.

Liang Xiao is currently a professor at the School of Computer Science, Nanjing University of Science and Technology (NUST). He received his PhD in computer science from NUST, China, in 2004. He has authored two books and around 100 technical articles in refereed journals, including the IEEE Transactions on Image Processing, the IEEE Transactions on Geoscience and Remote Sensing, and Pattern Recognition. His main research areas include computer vision and pattern recognition.

Pengfei Liu is currently an assistant professor at the School of Computer Science, Nanjing University of Posts and Telecommunications. He received his BS degree in information and computation science from Hefei Normal University, Hefei, China, in 2011, and his PhD in computer science and technology from Nanjing University of Science and Technology, Nanjing, in 2016. His research interests include computer vision and remote sensing data processing.

Lili Huang is currently an associate professor at Guangxi University of Science and Technology, China. She received her PhD in pattern recognition and intelligent systems from Nanjing University of Science and Technology, Nanjing, Jiangsu, China, in 2012. Her research interests cover image processing, image modeling, and superresolution reconstruction.

Nan Zhou: Biography is not available.

Yang Xu received his PhD in computer science from the Nanjing University of Science and Technology (NUST), Nanjing, China, in 2016. Currently, he is working as an assistant professor in Nanjing University of Science and Technology. His research interests are in the area of hyperspectral image classification, image processing, and machine learning.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Songze Tang, Liang Xiao, Pengfei Liu, Lili Huang, Nan Zhou, and Yang Xu "Pansharpening via sparse regression," Optical Engineering 56(9), 093105 (28 September 2017). https://doi.org/10.1117/1.OE.56.9.093105

Received: 13 April 2017; Accepted: 16 August 2017; Published: 28 September 2017

Access the abstract

JOURNAL ARTICLE
13 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 6 scholarly publications.

Explore citations on Lens.org

KEYWORDS

Associative arrays

Lawrencium

Image fusion

Wavelets

Spatial resolution

Optical engineering

Distortion

1.

Introduction

Fig. 1

2.

Pansharpening via Mapping of Sparse Coefficients

2.1.

Dictionary Learning and the Ridge Regression of Intrapatches

Eq. (1)

Eq. (2)

Fig. 2

Eq. (3)

Eq. (4)

Eq. (5)

Eq. (6)

Eq. (7)

Eq. (8)

Eq. (9)

Eq. (10)

Eq. (11)

Eq. (12)

2.2.

Learning Interpatch Regression Mapping Based on Elastic-Net Model

Eq. (13)

Fig. 3

Eq. (14)

Eq. (15)

Eq. (16)

Eq. (17)

Eq. (18)

Eq. (19)

2.3.

High-Resolution Multispectral Image Reconstruction

Eq. (20)

Eq. (21)

Eq. (22)

Eq. (23)

3.

Experimental Results and Analysis

3.1.

Parameter Analysis

Fig. 4

Fig. 5

Table 1

Table 2

Table 3

3.2.

Effects of Patch Size

Fig. 6

Fig. 7

3.3.

Effects of Dictionary Size

Fig. 8

Fig. 9

Fig. 10

Table 4

3.4.

Simulation Results and Analysis

Fig. 11

Table 5

Fig. 12

3.5.

Experiments on Real Data

Fig. 13

Fig. 14

Table 6

Table 7

3.6.

Time Consumption

4.

Conclusion

Acknowledgments

References

Biography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years