Imaging through scattering media using semidefinite programming

Hui Chen; Yesheng Gao; Xingzhao Liu; Zhixin Zhou

doi:10.1117/1.JBO.24.3.031016

29 November 2018 Imaging through scattering media using semidefinite programming

Hui Chen, Yesheng Gao, Xingzhao Liu, Zhixin Zhou

Author Affiliations +

Journal of Biomedical Optics, Vol. 24, Issue 3, 031016 (November 2018). https://doi.org/10.1117/1.JBO.24.3.031016

Abstract

A clear image of an observed object may deteriorate into unrecognizable speckle when encountering heterogeneous scattering media, thus it is necessary to recover the object image from the speckle. A method combining least square and semidefinite programming is proposed, which can be used for imaging through scattering media. The proposed method consists of two main stages, that is, media scattering characteristics (SCs) estimation and image reconstruction. SCs estimation is accomplished through LS concept after establishing a database of known object-and-speckle pairs. Image reconstruction is realized by solving an SDP problem to obtain the product of the unknown object image and its Hermitian transposition. Finally, the unknown object image can be reconstructed by extracting the largest rank-1 component of the product. Structural similarity (SSIM) index is employed as a performance indicator in speckle prediction and image reconstruction. Numerical simulations and physical experiments are performed to verify the feasibility and practicality of the proposed method. Compared with the existing phase shift interferometry mean square optimization method and the single-shot phase retrieval algorithm, the proposed method is the most precise to obtain the best reconstruction results with highest SSIM index value. The work can be used for exploring the potential applications of scattering media, especially for imaging through turbid media in biomedical, scattering property measurement, and optical image encryption.

1. Introduction

Light suffers from multiple scattering when propagating through heterogeneous media, which is a fundamental problem in practical applications ranging from physics, optics, and biological, to telecommunication and electromagnetism.¹^–⁷ The presence of scattering media makes the imaging result always a speckle pattern without any recognizable information, rather than the expected object image, limiting the developments of imaging in aforementioned fields.

In recent years, several methods have been proposed to reconstruct objects from speckles, such as the phase-shift interferometry mean square optimization (PSIMSO) method, the single-shot phase retrieval (SPR) algorithm.⁸^–¹⁴ Imaging with the PSIMSO usually utilizes the four-step phase-shift interferometry (PSI) to measure the scattering characteristic (SC) of scattering media and uses mean square optimization (MSO) to reconstruct the object image. To measure the SC, the required inputs are four times of the object image’s degrees of freedom. Thus, the PSI may not be much applicable in practical applications due to the fact that measuring SC with PSI is time-consuming.¹⁰^,¹¹^,¹⁵ The SPR avoids the SC measurement process and extract the object image from estimated Fourier spectrum.¹⁶^,¹⁷ In this method, the Fourier magnitude of the object image is calculated from the autocorrelation of captured speckle pattern, whereas the Fourier phase of which is estimated through hybrid input–output phase retrieval algorithm. However, the reconstruction results of SPR are always lack details, especially the ambiguous profile and low contrast.¹⁴

To address this situation, a method combining least square and semidefinite programming (LSSDP) is proposed for two-dimensional (2-D) imaging through scattering media in this paper. To improve efficiency and quality of image reconstruction when only magnitude of speckle field can be accessible,¹⁸^–²¹ the SC of media is first estimated through least square (LS) method after establishing a database of known object-and-speckle pairs. Once the SC is estimated, the lift convex optimization is utilized to obtain the product of the unknown object image and its Hermitian transposition, by solving a semidefinite programming (SDP) problem. Then the unknown object image can be reconstructed by extracting the largest rank-1 component of the product. Structural similarity (SSIM) index is employed as a performance indicator of the SC estimation and image reconstruction.¹⁹ Simulation and experimentation results validate that the proposed LSSDP outperforms than the PSIMSO and SPR in general. The work is expected to improve the image reconstruction quality for practical applications such as imaging through turbid media in biomedical,²²^–²⁴ and scattering property measurement of certain media,²⁴^,²⁵ such as fog and optical image encryption.²⁶^,²⁷

The rest of this paper is organized as follows: Sec. 2 introduces the principles, including experimental principle and imaging principle. Section 3 demonstrates simulation and experimentation results, i.e., comparison results between LSSDP and PSIMSO, as well as those between LSSDP and SPR. Finally, some conclusions are drawn in Sec. 4.

2. Principle

In this section, the experimental principle and imaging principle are introduced, respectively.

2.1.

Experimental Principle

Figure 1 shows the experimental setup used for imaging through scattering media. In our case, the scattering media is a 220 grit ground-glass diffuser (DG10-220-MD, Thorlabs). The monochromatic continuous wave laser (wavelength: 532 nm, output power: 15 mW) is served as a light source, sampled by a $20 - μ m$ pinhole and then expanded by a Fourier lens ( $f = 250 mm$ ). After that, the laser beam is spatially modulated by an amplitude-type Spatial Light Modulator (SLM, HES6001, Holoeye). The shaped beam is focused on the diffuser using the microscope objective ${MO}_{1}$ . The microscope objective ${MO}_{2}$ images the point (that is behind the diffuser) onto a complementary metal oxide semiconductor camera. Due to the presence of the scattering diffuser, only an unrecognizable speckle pattern (see Fig. 1) can be captured on camera. Although the speckle pattern seems to be unrecognizable at first glance, it contains adequate information to reconstruct the object image actually.

Fig. 1

Experimental principle.

Because of the imaging device limitation, only the magnitude of output speckle pattern could be recorded. In the paper, the input image vector loaded on SLM is denoted as $E^{in}$ , the SC of the diffuser is marked with $H$ , and the speckle pattern vector is represented as $E^{out}$ . $H$ is of dimension $M \times N$ , with $M$ and $N$ are the numbers of pixels contained by camera and SLM, respectively. The imaging through scattering media process can be expressed as

Eq. (1)

find E^{in} s . t . E^{out} = H \cdot E^{in} .

The imaging principle of the proposed method is illustrated next.

2.2.

Imaging Principle

In the proposed method, to reconstruct the original object image from speckle pattern, the first step is estimating the SC of the diffuser. Once the SC is estimated, the lift convex optimization can be applied to accomplish image reconstruction.²⁸^,²⁹

As for the SC estimation, the LS instead of the PSI is adopted. A set of different known object images should be modulated onto SLM one by one while the corresponding output speckle patterns can be collected. The main idea of the SC estimation can be expressed as

Eq. (2)

find H s . t . E^{out} = H \cdot E^{in} .

Equation (2) can be reformatted as

Eq. (3)

\min_{H} {‖ E^{out} - H \cdot E^{in} ‖}_{2}^{2} .

With the statements above, the SC of the diffuser can be estimated. Lift convex optimization is then utilized to reconstruct the unknown object image.

Considering the image reconstruction problem described in Eq. (1), where the subject is nonconvex, lift convex optimization is utilized to translate this nonconvex problem into convex one. The square of the $i^{'}$ th element of $E^{out}$ can be expressed as the square of magnitude of the inner product between the transpose of the $i^{'}$ th row vector of matrix $H$ and the input signal $E^{in}$ , namely

Eq. (4)

{(E_{i}^{out})}^{2} = {| ⟨ h_{i}, E^{in} ⟩ |}^{2} = T r [h_{i} h_{i}^{H} E^{in} {(E^{in})}^{H}] = T r (H H_{i} E E), i = 1,2, \dots, M,

where

E_{i}^{out}

is the

i^{'}

th element of

E^{out}

,

h_{i}

is the transpose of

i^{'}

th row vector of

H

,

⟨ h_{i}, E^{in} ⟩

is the inner product of

h_{i}

and

E^{in}

,

T r (\cdot)

stands for Trace,

{(\cdot)}^{H}

means the Hermitian transpose,

H H_{i} = h_{i} h_{i}^{H}

and

E E = E^{in} {(E^{in})}^{H}

are symmetric matrix and symmetric positive semidefinite matrix, respectively. Note that, the rank of

E E

is always 1.

Based on Eq. (4), the problem described in Eq. (1) could then become

Eq. (5)

find E E s . t . T r (H H_{i} E E) = {(E_{i}^{out})}^{2}, i = 1,2, \dots, M rank (E E) = 1 E E \geq 0,

where the constraint

E E \geq 0

refers to its positive semidefinite characteristic.

As the second constraint of Eq. (5) is nonconvex, the problem can be reformatted as

Eq. (6)

\min rank (E E) s . t . T r (H H_{i} E E) = {(E_{i}^{out})}^{2}, i = 1,2, \dots, M E E \geq 0 .

Replacing the NP-hard problem of minimizing rank with minimizing trace, then the problem can be rewritten as²⁹

Eq. (7)

\min T r (E E) s . t . T r (H H_{i} E E) = {(E_{i}^{out})}^{2}, i = 1,2, \dots, M E E \geq 0 .

Once Eq. (7) is solved, the unknown object image can be estimated with the largest rank-1 component of $E E$ .

The whole imaging through scattering media process can be described.

As shown in Fig. 2, after collecting adequate known object-and-speckle pairs, SC of media can be estimated with the LS method. Then with the measured speckle pattern of an unknown object, the product of the unknown object image and its Hermitian transposition can be obtained and then resolved using SDP. Then, the unknown object can be reconstructed through extracting the largest rank-1 component of the product. Next, the employed indicators for image reconstruction fidelity evaluation are introduced.

Fig. 2

Block diagram of the proposed LSSDP method.

2.3.

Image Reconstruction Fidelity Evaluator

The employed performance indicator, SSIM index (calculated on the whole image), is defined as follows:³⁰

Eq. (8)

SSIM (X, Y) = {[\frac{2 μ_{X} μ_{Y} + c_{1}}{μ_{X}^{2} + μ_{Y}^{2} + c_{1}}]}^{α} {[\frac{2 σ_{X} σ_{Y} + c_{2}}{σ_{X}^{2} + σ_{Y}^{2} + c_{2}}]}^{β} {[\frac{σ_{X, Y} + c_{3}}{σ_{X} σ_{Y} + c_{3}}]}^{γ},

where

μ_{X}

and

μ_{Y}

are the mean values of image

X

and

Y

,

σ_{X}

and

σ_{Y}

are the standard deviations of

X

and

Y

,

σ_{X, Y}

represents the covariance of

X

and

Y

,

c_{1}

,

c_{2}

, and

c_{3}

are the small constants to avoid instability when the three denominators tend to zero,

α

,

β

, and

γ

are all larger than zero and are used to adjust the relative proportion of three multipliers.

Set weights $α$ , $β$ , and $γ$ all to 1, in addition to above definitions: $c_{3} = c_{2} / 2$ , then Eq. (8) can be reformatted as

Eq. (9)

SSIM (X, Y) = \frac{(2 μ_{X} μ_{Y} + c_{1}) (2 σ_{X Y} + c_{2})}{(μ_{X}^{2} + μ_{Y}^{2} + c_{1}) (σ_{X}^{2} + σ_{Y}^{2} + c_{2})},

where

c_{1}

and

c_{2}

are set to default values as 0.0001 and 0.0009, respectively. When the image

X

is identical to image

Y

, the SSIM reaches value 1.

In the SC estimation performance evaluation, the SSIM index is calculated between predicted speckle and system output speckle. In the image reconstruction performance evaluation, the SSIM index is calculated between the reconstructed image and original input object image. Higher SSIM index value means better SC estimation or image reconstruction performance. To verify the feasibility and practicality of the proposed LSSDP, simulation and experimentation results of comparisons with PSIMSO and SPR are demonstrated, respectively.

3. Simulation and Experimentation Results

In this section, the performance of the LSSDP is evaluated by comparisons with PSIMSO and SPR through numerical simulations and physical experiments.

3.1.

Comparisons with Phase Shift Interferometry Mean Square Optimization

Numerical simulation and physical experimentation comparisons between the proposed LSSDP and the PSIMSO are demonstrated in this section. Simulation results are presented in Sec. 3.1.1, whereas experimentation results are detailed in Sec. 3.1.2.

3.1.1.

Simulation comparisons

Circle Gaussian distribution model is introduced to model the SC of the diffuser. A circular Gaussian distribution matrix (CGDM) is simulated for both the LSSDP and the PSIMSO.¹⁰^,¹¹^,³¹ The assumption that noise level can be precisely estimated for the PSIMSO with correlation-maximization method is made to ensure that our evaluations are only targeted at the image reconstruction performance of each method.¹¹^,¹²

In simulation, symbol “O” is served as the object image to be reconstructed. The reconstructed images with the PSIMSO and the LSSDP under different noise levels are shown in Fig. 3. The considered signal-to-noise (SNR) levels are 50, 40, 30, and 20 dB, respectively.

Fig. 3

Numerical simulated imaging results of symbol “O” using (a)–(d) PSIMSO and (e)–(h) LSSDP under several noise levels.

The first row in Fig. 3 shows the reconstructed images of the PSIMSO, and the second row shows those of the LSSDP. Both methods perform worse along with the increasing of noise level, but the LSSDP always works better than the PSIMSO under the same noise level. When the SNR is 20 dB (see the rightmost column in Fig. 3), the PSIMSO whose reconstruction contains higher background noise performs worse than the LSSDP. To compare the noise robustness of these two methods much more intuitively, their image reconstruction performances evaluated with the SSIM index are shown in Fig. 4, where the green triangle line denotes the PSIMSO and the blue circle line represents the proposed LSSDP.

Fig. 4

Image reconstruction quality comparisons between PSIMSO and LSSDP under different noise levels, when symbol “O” serves as the image to be reconstructed. The considered SNR values are 50, 40, 30, 20, 10, 5, and 1 dB, respectively.

It can be clearly seen from Fig. 4 that these two methods have good performances under high SNR level, and the maximum SSIM index value can be up to 0.9994 (with LSSDP) and 0.7239 (with PSIMSO) when SNR is 50 dB. The proposed LSSDP always performs better than the PSIMSO under a certain noise level. In addition, for both methods, when SNR is $< 5 dB$ , the SSIM index values decrease slowly along with the decreasing of SNR.

To compare performance of the PSIMSO and the LSSDP in a further step, simulated reconstruction results of the PSIMSO and the LSSDP are summarized in Fig. 5 when a complex biomedical DICOM image served as an input object image of the scattering system. The considered SNR levels are still 50, 40, 30, and 20 dB, respectively.

Fig. 5

Numerical simulated imaging results of a complex DICOM image using (a)–(d) PSIMSO and (e)–(h) LSSDP under several noise levels.

In the reconstruction of the complex DICOM image, the LSSDP makes a near-perfect reconstruction with SSIM 0.9490, where the details can be clearly and apparently distinguished from the reconstructed image [see Fig. 5(e)]. Both the performances of the PSIMSO and the LSSDP degrade with the increasing of the noise level, but the proposed LSSDP performs better than the PSIMSO relatively. When SNR is 50 dB, the reconstruction of PSIMSO [with SSIM 0.7282, see Fig. 5(a)] is already along with some ambiguities, and the ambiguities become more severe after increasing the noise level. When SNR is 20 dB, the reconstruction with LSSDP [with SSIM 0.6207, see Fig. 5(h)] is also along with some ambiguous background, but the phenomenon is slighter than that in the reconstruction with PSIMSO [with SSIM 0.4707, see Fig. 5(d)], where some details are almost not able to distinguish. Comparing with reconstructions of symbol “O” in Fig. 3, the LSSDP is found to be more applicable to reconstruct sparse object (such as the symbol “O,” with lower background) under relatively high SNR level.

To compare performance of the PSIMSO and the LSSDP more intuitively, their image reconstruction performances (when the DICOM image serves as target image to be reconstructed) evaluated with SSIM index are shown in Fig. 6, where the green triangle line denotes the PSIMSO and the blue circle line represents the proposed LSSDP.

Fig. 6

Image reconstruction quality comparisons between PSIMSO and LSSDP under different noise levels, when the complex DICOM image serves as the image to be reconstructed. The considered SNR values are 50, 40, 30, 20, 10, 5, and 1 dB, respectively.

In Fig. 6, for both methods, when SNR is $> 5 dB$ , the SSIM index values decrease significantly along with the decreasing of SNR, whereas the SSIM index values decrease slowly along with the decreasing of SNR in the case of SNR $< 5 dB$ . Comparing with Fig. 4 (where the symbol “O” serves as the target image to be reconstructed), when SNR is $< 5 dB$ , the calculated SSIM index values of the considered two methods here are relatively low than those in Fig. 4, this may due to the fact that the tested DICOM image is more complex than the symbol “O” so that the backgrounds of reconstructions contribute to the lower SSIM indices.

From the above-simulated results, one can see that the proposed LSSDP can effectively realize image reconstruction and performs better than the PSIMSO under the same SNR case. In addition, both the PSIMSO and the LSSDP perform better in the reconstruction of a simple sparse object than in the reconstruction of a complex object, especially the LSSDP. Through comparison, the LSSDP is more applicable for its capability in reserving image details, and the superior is more obvious in the reconstruction of a complex object with much detail. In the following, experiments are conducted to compare the SC estimation³¹^,³² and image reconstruction performance separately in detail.

3.1.2.

Experimentation comparisons

In experiment, the SC is measured with LS (SC with LSSDP) and PSI (SC with PSIMSO), respectively. The image reconstruction performances are compared separately from the SC estimation performances. Both the SC estimation performance and image reconstruction performance are all evaluated through SSIM index value. The comparison principles are shown in Fig. 7.

Fig. 7

Experimental comparison principles between PSIMSO and LSSDP.

The SC estimation is evaluated through speckle pattern prediction, that is, the SC whose predicted speckle is closer to the system actual output speckle is treated as the more precise estimated ones. Then, the more precise SC is chosen to be the one that is going to be used in lateral image reconstruction. Here, the better SC is the one measured with LS (which can be seen from the higher speckle pattern prediction capability in Fig. 8). So, the imaging result with LS-estimated SC and MSO reconstruction method is also considered, namely LSMSO.

Fig. 8

Experimental imaging results of PSIMSO and LSSDP when symbol “O” is served as object to be reconstructed. (a) The input image of symbol “O,” (b) the output speckle pattern, (c) the SC estimated with PSIMSO, (d) the SC estimated with LSSDP, (e) the imaging result with PSIMSO, (f) the imaging result with LSMSO, (g) the imaging result with LSSDP, (h)–(n) as in (a)–(g) but for the biomedical DICOM image.

Calibration images used to estimate the SC of the diffuser with LSSDP are randomly chosen. The chosen calibration images are all resized to $64 \times 64$ , to be modulated on the central of the SLM (see Fig. 1) with $1920 \times 1080$ resolution. Using the optical setup shown in Fig. 1, the captured speckle patterns are all downsampled to $64 \times 64$ . In that case, the dimension of the SC is $64^{2} \times 64^{2}$ , and $64^{2} \times 64^{2}$ reshaped Hadamard vectors are necessary for the PSIMSO. In this comparison, symbol “O” and the biomedical DICOM image still serve as the images to be reconstructed. In Fig. 8, the SC estimation performance and image reconstruction performance of LSSDP and PSIMSO with the same number of calibration images are compared.

The higher SSIM values of LSSDP [Fig. 8(d) with SSIM 0.8398, Fig. 8(k) with SSIM 0.7349] in a speckle pattern prediction indicate that the LSSDP estimates the SC of the diffuser more precisely than PSIMSO [Fig. 8(c) with SSIM 0.5365, Fig. 8(j) with SSIM 0.4568]. So, the reconstructions using LSMSO are also considered [see Figs. 8(f) and 8(m)]. As for the difference in speckle pattern prediction of each method, the input images may be responsible for this. Because, according to Eq. (9), the mean value and the variance of referenced input images may influence the calculation of the SSIM index. The images reconstructed with LSSDP [Fig. 8(g) with SSIM 0.4856, Fig. 8(n) with SSIM 0.4394] are better than those reconstructed using the PSIMSO [Fig. 8(e) with SSIM 0.2549, Fig. 8(l) with SSIM 0.2384], and also better than those reconstructed using LSMSO [Fig. 8(f) with SSIM 0.3360, Fig. 8(m) with SSIM 0.2731]. In a word, LSSDP performs better than PSIMSO not only in the SC estimation, but also in the image reconstruction. In addition, in Figs. 8(e) and 8(l), the profile of symbol “O” is generally recovered while the details of the DICOM image are difficult to distinguish, so the PSIMSO may not be very suitable for imaging complex objects. Meanwhile, the LSSDP is also more feasible and adaptable for imaging sparse object but can be further improved in the reconstruction of complex objects.

Apart from the SC estimation ability and image reconstruction ability, the singular value distribution (SVD) is compared as well. A comparison of SVD based on SCs, estimated with the PSIMSO, the LSSDP, and simulated CGDM, is shown in Fig. 9. The SVD of SC estimated with LSSDP is closer to that of CGDM, than that of SC estimated with the PSIMSO, which also verifies the principles discussed in Sec. 2. The results explain why the proposed LSSDP performs better than the PSIMSO and also validate the feasibility and practicality of using CGDM in simulation. Note that, as for CGDM, the statistical distribution $ρ (λ)$ of the normalized singular values $λ$ follows quarter-circle law and can be formatted as⁵

Eq. (10)

ρ (λ) = \frac{1}{π} \sqrt{4 - λ^{2}} .

Fig. 9

SVD comparison among CGDM (red solid line), SC estimated with PSIMSO (green triangle line), and SC estimated with LSSDP (blue circle line).

To further show that it is reasonable to use CGDM to model the SC in simulation, a column coherence comparison between CGDM and experimental obtained SCs is shown in Fig. 10. The coherence is calculated from the maximal collinearity between the columns of a matrix. For all matrices, column coherence decreases along with an increase in the measurement numbers. When measurement finishes, lower coherence is found in LSSDP-estimated SC than PSIMSO-estimated SC.

Fig. 10

Column coherence comparison among CGDM (red line), SC estimated with PSIMSO (green line), and SC estimated with LSSDP (blue line).

From the figures above, some conclusions can be drawn. First, both the proposed LSSDP and the PSIMSO can reconstruct the original input images under the same experimental setup as Fig. 1. Second, the LSSDP can achieve better SC estimation and image reconstruction performance than the PSIMSO with the same number of measurements. In general, the proposed LSSDP works better than the PSIMSO. In addition, the SVD of LSSDP measured SC is closer to quarter-circle law, and the measured SC owns lower column coherence than PSIMSO, which inversely explains the phenomenon in Fig. 8. Third, the LSSDP performs better in a simple sparse object condition. The imaging principle of LSSDP may be responsible for the phenomenon as details of the tested DICOM image may be low-contrast in intensity and thus make it hard to distinguish the corresponding component from noise when extracting the largest rank-1 component of the product of the unknown object image and its Hermitian transposition.

3.2.

Comparisons with Single-Shot Phase Retrieval

SPR can be used to reconstruct 2-D images from their Fourier magnitude, under constraints such as nonnegativity and support region.¹⁴ To assure faithful reconstruction of object images, several restarts of the algorithm are performed with different random initial Fourier phases.¹⁴ The result having the closest autocorrelation function to that of captured speckle (highest SSIM index value) is chosen as the final reconstructed image.

The proposed LSSDP is compared with the SPR to verify the image reconstruction performance. Both simulation and experimentation results are detailed in this section.

3.2.1.

Simulation comparisons

In simulation, symbol “E” and a magnetic resonance image are served as the object image to be reconstructed. The reconstructed images of symbol “E” with SPR and LSSDP under different noise levels are shown in Fig. 11. The considered SNR levels are 50, 40, 30, and 20 dB, respectively.

Fig. 11

Numerical simulated imaging results of symbol “E” using (a)–(d) SPR and (e)–(h) LSSDP under several noise levels.

The first row in Fig. 11 shows the reconstructed images of the SPR, and the second row depicts those of the LSSDP. The performances of both methods degrade along with the increasing of the noise level, but the LSSDP always works better than SPR under the same noise level. In Fig. 11, both the LSSDP and the SPR realize the object image reconstruction. In addition, under a relatively high noise level, there is blurred boundary phenomenon in SPR to some extent, while the phenomenon would not occur in LSSDP. To compare the noise robustness of these two methods much more intuitively, their image reconstruction performances evaluated with SSIM index are shown in Fig. 12, where the green triangle line denotes the SPR and the blue circle line represents the proposed LSSDP.

Fig. 12

Image reconstruction quality comparisons between SPR and LSSDP under different noise levels, when symbol “E” serves as the image to be reconstructed. The considered SNR values are 50, 40, 30, 20, 10, 5, and 1 dB, respectively.

In Fig. 12, the two methods have good performances under high SNR, and the maximum SSIM index value can be up to 0.9573 (using the LSSDP) and 0.6491 (using the SPR) when SNR is 50 dB. The proposed LSSDP always performs better than the SPR under a certain SNR level. In addition, both methods are influenced and the SSIM index values are decreasing sharply until the SNR decreases to 10 dB. The decreasing rate of the LSSDP is higher than that of SPR, which means that the SPR owns more stable performance, but the LSSDP performs better in general.

To compare performance of the SPR and the LSSDP in a further step, simulated reconstruction results of the SPR and the LSSDP are shown in Fig. 13 when a magnetic resonance image served as an input object image of the scattering system. The considered SNR levels are still 50, 40, 30, and 20 dB, respectively.

Fig. 13

Numerical simulated imaging results of a magnetic resonance image using (a)–(d) SPR and (e)–(h) LSSDP under several noise levels.

In the reconstructions of the complex magnetic resonance image, the LSSDP makes a satisfying reconstruction with SSIM 0.8969, where the profile can be clearly and apparently distinguished from the reconstructed image [see Fig. 13(e)]. In addition, the ambiguity in Fig. 13(e) may be responsible for the lower SSIM index values, compared with the former simulated results of LSSDP when SNR is 50 dB [see Figs. 3(e), 5(e), and 11(e) for details]. The performances of the SPR and the LSSDP all degrade along with the increasing of the noise level, especially the LSSDP. But under a certain SNR level, the proposed LSSDP performs better than the SPR relatively. When SNR is 20 dB, even there exist high background noises in the reconstruction of the LSSDP [see Fig. 13(h)], details of the profile are still remained, which means that the background may be filtered by setting an appropriate threshold. As for the SPR, the reconstruction result when SNR is 50 dB [with SSIM 0.5591, see Fig. 13(a)] is already along with some ambiguities, and the phenomenon becomes more severe after increasing the noise level. When SNR is 20 dB, the reconstruction of LSSDP [with SSIM 0.5094, see Fig. 13(h)] is also along with ambiguous background, but the phenomenon is slighter than that in the reconstruction of SPR [with SSIM 0.3946, see Fig. 13(d), where some details are almost not able to distinguish]. Note that, the original magnetic resonance image is actually not sparse (filled with low gray level intensity), and this is the main reason why the reconstructions of the SPR and the LSSDP all contain some ambiguities more or less. Comparing with reconstructions in Fig. 5 where complex but low-contrast image (i.e., the DICOM image) serves as the target image to be reconstructed, the LSSDP is found to be more applicable to reconstruct the object with lower background but higher contrast (such as the magnetic resonance image here).

To compare performance of the SPR and the LSSDP more intuitively, their image reconstruction performances evaluated with SSIM index are shown in Fig. 6, where the green triangle line denotes the SPR and the blue circle line represents the proposed LSSDP.

In Fig. 14, for both methods, when SNR is $> 5 dB$ , the SSIM index values decrease significantly along with the decreasing of SNR, whereas the SSIM index values decrease slowly along with the decreasing of SNR once SNR is $< 5 dB$ . The phenomenon is especially obvious for the LSSDP, which validates that the LSSDP is more applicable to high SNR level conditions. As for the SPR, when SNR is $< 5 dB$ , the performance is not satisfying, which may be caused by the imprecisely estimated Fourier amplitude under high noise level conditions.

Fig. 14

Image reconstruction quality comparisons between SPR and LSSDP under different noise levels, when a magnetic resonance image serves as the image to be reconstructed. The considered SNR values are 50, 40, 30, 20, 10, 5, and 1 dB, respectivley.

From the above-simulated results, the proposed LSSDP realizes image reconstruction and performs better than the SPR under the same SNR case in general. In the next section, experiments are conducted to compare the image reconstruction performance.

3.2.2.

Experimentation comparisons

For practical comparison between LSSDP and SPR, not only physical experimental results are represented and analyzed but also the corresponding iteration convergence curve is given. In this comparison, symbol “OE” (where intensity of “E” is half of intensity of “O”) and the magnetic resonance image (tested in last section) serve as the images to be reconstructed. The reconstruction results of the SPR and the LSSDP are shown in Fig. 15.

Fig. 15

Experimental results of SPR and LSSDP. The first row shows results when symbol “OE” is served as object to be reconstructed: (a) the original object image, (b) the imaging result with SPR, and (c) the imaging result with LSSDP. (d)–(f) As in (a)–(c) but for the biomedical magnetic resonance image.

In Fig. 15, the SPR recovered the general profile of input images, but the blurred boundary phenomenon (mentioned in Sec. 3.2.1) becomes more severe in experiment than in simulation. The LSSDP not only recovered profile of input images but also some details. The backgrounds of reconstructions using the SPR are always more ambiguous than those of reconstructions using the LSSDP. In addition, for the two considered methods, the performance in reconstruction of a simpler symbol [see Fig. 15(a)] is better than that in the reconstruction of a complex object [see Fig. 15(d)], and the phenomenon is especially obvious for the LSSDP. In a word, the LSSDP performs better than the SPR with clearer profile, lower background noise, and higher contrast. In addition, the corresponding iteration converge curves are shown in Fig. 16.

Fig. 16

Iteration converge curves of SPR and LSSDP: (a) when symbol “OE” is served as object to be reconstructed and (b) when the biomedical magnetic resonance image is served as object to be reconstructed.

The SSIM values in Fig. 16 are calculated between autocorrelation spectrum of the input object image and that of reconstructed intermediate results. The SPR converges more quickly than the LSSDP in general. When the symbol “OE” is served as object to be reconstructed [see Fig. 16(a)], the SPR costs about 200 iterations to converge to SSIM 0.3954 [see Fig. 15(b)], whereas the LSSDP costs about 400 iterations to converge to SSIM 0.5679 [see Fig. 15(c)]. In addition, the SSIM of LSSDP is 0.5369 after 200 iterations. When the magnetic resonance image is served as an object to be reconstructed [see Fig. 16(b)], 200 iterations are still adequate for the SPR to converge to SSIM 0.2908 [see Fig. 15(e)], but more iterations (about 900 iterations) are needed for the LSSDP to converge to SSIM 0.4617 [see Fig. 15(f)].

The results show that the convergence of the SPR is almost irrelevant to the object (to be reconstructed) and the SPR can always converge at a high speed. As for the LSSDP, comparing to former experiments, the calculated SSIM values are relatively lower than before. The complex gray-scale images themselves should be responsible for the phenomenon as relatively low gray level would make it difficult to distinguish the corresponding component from noise when extracting the largest rank-1 component of the product of the unknown object image and its Hermitian transposition. The phenomenon demonstrates that, to some extent, the image reconstruction performance of the LSSDP is related to the complexity of the object to be reconstructed. That is, simpler sparse object would lead to a faster convergence rate and higher reconstruction fidelity, whereas complex nonsparse object with multigray scale would lead to slower convergence rate and lower reconstruction fidelity, and a not so satisfying result will be obtained when imaging a complex object.

The conclusion that the LSSDP is relatively more applicable than the SPR generally could be concluded. The differences existing in performance between simulations and experiments are mainly caused by the existence of noises that are composed of environment noise, noise coming from instruments, such as modulator or camera. Noise is usually unavoidable and nonnegligible in physical experiments and would lead biases to results but could be suppressed partly by choosing relative stable time or instruments for experiments. In our case, reconstruction performance may get improvements with an additional noise suppression process before extracting the largest rank-1 component of the product of the unknown object image and its Hermitian transposition. The noise suppression process is filtering noise or bias component through setting a threshold to singular value decomposition of the product in detail.

4. Conclusion

In this research, we have demonstrated a method for imaging through scattering media, that is, the LSSDP, where the SC of the scattering media is measured with LS algorithm and the image reconstruction is accomplished with lift convex optimization by solving an SDP problem. SSIM index is introduced to evaluate the SC estimation performance and image reconstruction performance. Feasibility and practicality are validated from both simulation and experimentation comparisons with PSIMSO and SPR, respectively. Comparisons with PSIMSO show that the lower column-correlated but more qualified SC (measured with the proposed LSSDP), as well as the more effective image reconstruction method SDP, all contribute to a much more highly structural-similar (with input) reconstruction result. Comparisons with SPR reveal that the proposed method performs better in image reconstruction with lower background and higher SSIM index values. However, the lower image reconstruction fidelity with complex biomedical objects shows that further efforts are needed to improve the image reconstruction fidelity of complex objects in practice. The work is expected to improve the image reconstruction quality for practical applications of imaging through turbid media in biomedical and biophotonics imaging.

Disclosures

The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (Grant No. 61601285).

References

1.

P. J. Keller et al., “Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy,” Science, 322 1065 –1069 (2008). https://doi.org/10.1126/science.1162493 SCIEAS 0036-8075 Google Scholar

2.

H. D. Bartunik and B Chance, “Structural biological applications of x-ray absorption, scattering, and diffraction,” (1986). Google Scholar

3.

D. I. Svergun, “Restoring low resolution structure of biological macromolecules from solution scattering using simulated annealing,” Biophys. J., 76 (6), 2879 –2886 (1999). https://doi.org/10.1016/S0006-3495(99)77443-6 BIOJAU 0006-3495 Google Scholar

4.

Y. Mizuno and K. Nakamura, “Experimental study of Brillouin scattering in perfluorinated polymer optical fiber at telecommunication wavelength,” Appl. Phys. Lett., 97 (2), 1475 (2010). https://doi.org/10.1063/1.3463038 APPLAB 0003-6951 Google Scholar

5.

A. Bossavit, “Simplicial finite elements for scattering problems in electromagnetism,” Comput. Methods Appl. Mechan. Eng., 76 (3), 299 –316 (1989). https://doi.org/10.1016/0045-7825(89)90062-5 Google Scholar

6.

M. B. Holcomb et al., “Investigating electric field control of magnetism with neutron scattering, nonlinear optics and synchrotron x-ray spectromicroscopy,” Int. J. Mod. Phys. B, 26 (10), 392 (2012). https://doi.org/10.1142/S0217979212300046 Google Scholar

7.

C. Amra, “Light-scattering from multilayer optics. 1. Tools of investigation,” J. Opt. Soc. Am. A, 11 (1), 197 –210 (1994). https://doi.org/10.1364/JOSAA.11.000197 JOAOD6 0740-3232 Google Scholar

8.

J. Bortolotti et al., “Non-invasive imaging through opaque scattering layers,” Nature, 491 232 –234 (2015). https://doi.org/10.1038/nature11578 Google Scholar

9.

I. M. Vellekoop and C. M. Aegerter, “Scattered light fluorescence microscopy: imaging through turbid layers,” Opt. Lett., 35 (8), 1245 –1247 (2010). https://doi.org/10.1364/OL.35.001245 OPLEDP 0146-9592 Google Scholar

10.

S. M. Popoff et al., “Image transmission through an opaque material,” (2010). https://doi.org/10.1038/ncomms1078 Google Scholar

11.

S. M. Popoff et al., “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett., 104 (10), 100601 –100604 (2010). https://doi.org/10.1103/PhysRevLett.104.100601 PRLTAO 0031-9007 Google Scholar

12.

S. M. Popoff et al., “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys., 13 (12), 123021 –123028 (2011). https://doi.org/10.1088/1367-2630/13/12/123021 NJOPFM 1367-2630 Google Scholar

13.

Y. Choi et al., “Overcoming the diffraction limit using multiple light scattering in a highly disordered medium,” Phys. Rev. Lett., 107 (2), 023902 –023906 (2011). https://doi.org/10.1103/PhysRevLett.107.023902 PRLTAO 0031-9007 Google Scholar

14.

O. Katz et al., “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photonics, 8 784 –790 (2014). https://doi.org/10.1038/nphoton.2014.189 NPAHBY 1749-4885 Google Scholar

15.

S. M. Popoff et al., “Exploiting the time-reversal operator for adaptive optics, selective focusing, and scattering pattern analysis,” Phys. Rev. Lett., 107 (26), 263901 –263906 (2011). https://doi.org/10.1103/PhysRevLett.107.263901 PRLTAO 0031-9007 Google Scholar

16.

E. G. Van Putten, A. Lagendijk and A. P. Mosk, “Nonimaging speckle interferometry for high-speed nanometer-scale position detection,” Opt. Lett., 37 (6), 1070 –1072 (2012). https://doi.org/10.1364/OL.37.001070 OPLEDP 0146-9592 Google Scholar

17.

S. Bahmani and J. Romberg, “Phase retrieval meets statistical learning theory: a flexible convex relaxation,” (2017). Google Scholar

18.

H. Ohlsson et al., “Compressive phase retrieval from squared output measurements via semidefinite programming,” in IFAC Symp. on System Identification, (2012). https://doi.org/10.3182/20120711-3-BE-2027.00415 Google Scholar

19.

D. Briers et al., “Laser speckle contrast imaging: theoretical and practical limitations,” J. Biomed. Opt., 18 (6), 066018 (2013). https://doi.org/10.1117/1.JBO.18.6.066018 JBOPFO 1083-3668 Google Scholar

20.

I. Lee, N. Tian and J. Romberg, “Fast and guaranteed blind multichannel deconvolution under a bilinear system model,” (2016). Google Scholar

21.

J. Mertz, “Optical sectioning microscopy with planar or structured illumination,” Nat. Methods, 8 (10), 811 –819 (2011). https://doi.org/10.1038/nmeth.1709 1548-7091 Google Scholar

22.

N. Ghosh, A. Banerjee and J. Soni, “Turbid medium polarimetry in biomedical imaging and diagnosis,” Eur. Phys. J. Appl. Phys., 54 (3), 184 –187 (2011). https://doi.org/10.1051/epjap/2011110017 EPAPFV 1286-0042 Google Scholar

23.

A. E. Genina, N. A. Bashkatov and V. V. Tuchin, “Tissue optical immersion clearing,” Expert Rev. Med. Dev., 7 (6), 825 –842 (2010). https://doi.org/10.1586/erd.10.50 1743-4440 Google Scholar

24.

R. Fiolka, K. Si and M. Cui, “Complex wavefront corrections for deep tissue focusing using low coherence backscattered light,” Opt. Express, 20 (15), 16532 –16543 (2012). https://doi.org/10.1364/OE.20.016532 OPEXFF 1094-4087 Google Scholar

25.

I. A. Newman and K. J. Webb, “Imaging optical fields through heavily scattering media,” Phys. Rev. Lett., 113 (26), 263903 –263908 (2014). https://doi.org/10.1103/PhysRevLett.113.263903 PRLTAO 0031-9007 Google Scholar

26.

N. Rawat et al., “Compressive sensing based robust multispectral double-image encryption,” Appl. Opt., 54 (7), 1782 –1793 (2015). https://doi.org/10.1364/AO.54.001782 APOPAI 0003-6935 Google Scholar

27.

I. Li et al., “Compressive optical image encryption,” Sci. Rep., 5 10374 (2015). https://doi.org/10.1038/srep10374 SRCEC3 2045-2322 Google Scholar

28.

E. J. Candes, T. Strohmer and V. Voroninski, “Phaselift: exact and stable signal recovery from magnitude measurements via convex programming,” Commun. Pure Appl. Math., 66 (8), 1241 –1274 (2013). https://doi.org/10.1002/cpa.21432 CPMAMV 0010-3640 Google Scholar

29.

A. Aghasi and J. Romberg, “Learning shapes by convex composition,” (2016). Google Scholar

30.

Z. Wang et al., “Image quality assessment: from error visibility to structuralsimilarity,” IEEE Trans. Image Process., 13 (4), 600 –612 (2004). https://doi.org/10.1109/TIP.2003.819861 IIPRE4 1057-7149 Google Scholar

31.

R. Vershynin, “Introduction to the non-asymptotic analysis of random matrices,” (2011). Google Scholar

32.

A. Drémeau et al., “Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques,” Opt. Express, 23 (9), 11898 –11911 (2015). https://doi.org/10.1364/OE.23.011898 OPEXFF 1094-4087 Google Scholar

Biography

Hui Chen received her BE degree in School of Electronic Engineering from Xidian University in 2015. Currently, she is a PhD candidate of information and communication engineering at Shanghai Jiao Tong University. Her current research interests include novel optical imaging technology, pattern recognition, and optical signal processing.

Yesheng Gao received his BE degree in information engineering in 2005 and his ME degree in communication and information systems in 2008, both from Nanjing University of Aeronautics and Astronautics, and his PhD in signal and information processing from Shanghai Jiao Tong University in 2013. Currently, he is an assistant professor at Shanghai Jiao Tong University. His research interests include novel optical imaging technology, pattern recognition, adaptive optics, and signal processing.

Xingzhao Liu received his BE and ME degrees both from Harbin Institute of Technology, and his PhD in electronic engineering in 1998 from the University of Tokushima, Japan. Currently, he is a professor and dean at the Department of Electronics at Shanghai Jiao Tong University. His research interests include signal processing, and novel sensor technology.

Zhixin Zhou is currently a professor at Space Engineering University, and an academician of Chinese Academy of Sciences. His current research interests include signal processing, remote sensing, and sensor technology and its applications.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Hui Chen, Yesheng Gao, Xingzhao Liu, and Zhixin Zhou "Imaging through scattering media using semidefinite programming," Journal of Biomedical Optics 24(3), 031016 (29 November 2018). https://doi.org/10.1117/1.JBO.24.3.031016

Received: 29 June 2018; Accepted: 5 November 2018; Published: 29 November 2018

Access the abstract

JOURNAL ARTICLE
11 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 1 patent.

Explore citations on Lens.org

KEYWORDS

Signal to noise ratio

Image restoration

Scattering media

Computer programming

Reconstruction algorithms

Image analysis

Speckle

1.

Introduction

2.

Principle

2.1.

Experimental Principle

Fig. 1

Eq. (1)

2.2.

Imaging Principle

Eq. (2)

Eq. (3)

Eq. (4)

Eq. (5)

Eq. (6)

Eq. (7)

Fig. 2

2.3.

Image Reconstruction Fidelity Evaluator

Eq. (8)

Eq. (9)

3.

Simulation and Experimentation Results

3.1.

Comparisons with Phase Shift Interferometry Mean Square Optimization

3.1.1.

Simulation comparisons

Fig. 3

Fig. 4

Fig. 5

Fig. 6

3.1.2.

Experimentation comparisons

Fig. 7

Fig. 8

Eq. (10)

Fig. 9

Fig. 10

3.2.

Comparisons with Single-Shot Phase Retrieval

3.2.1.

Simulation comparisons

Fig. 11

Fig. 12

Fig. 13

Fig. 14

3.2.2.

Experimentation comparisons

Fig. 15

Fig. 16

4.

Conclusion

Disclosures

Acknowledgments

References

Biography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years