Open Access Paper
26 June 2017 Imaging cell clusters and tissue using learning tomography
M. Hasani Shoreh, A. Goy, J. Lim, U. Kamilov, M. Unser, D. Psaltis
Author Affiliations +
Abstract
We present an experimental comparison of optical tomography techniques: Learning Tomography (LT), an iterative method based on beam propagation model and diffraction tomography (DT) based Born and Rytov approximation. Imaging experiments were performed on yeast cells clusters fixed in a transparent agarose gel. In particular, we compare the LT with a similar iterative but linear method based on Rytov approximation.

1.

INTRODUCTION

Over the past decade, optical tomography has attracted attention due to the promise for three-dimensional imaging of biological processes at the cell level [1-7]. The working principle of the technique is exposed in the seminal paper by Emil Wolf [8] that proposes a direct formula to calculate the refractive index distribution of the object from measurements of the scattered field. This formula, which we refer to as the Wolf transform, is based on the first Born approximation and is a linear map from the refractive index to the scattered field. The corollary is that the Wolf transform only accounts for single scattering and is thus restricted to objects having weak refractive index contrast. Significant improvement in image quality can be obtained by using the Rytov approximation [9] in conjunction with the Wolf transform [10]. However, this approximation solution is still based on a linearized scattering equation and is also a single scattering model. The Rytov and Born approximations yield satisfactory results for objects that are small and with moderate refractive index contrast such as a single cell or small group of cells. For thicker object, multiple scattering can no longer be neglected.

In recent publications, we proposed Learning Tomography (LT) as a new tomography technique, based on machine learning concepts and a propagation model, namely the beam propagation method (BPM) that accounts for multiple scattering [11, 12]. Here, we present recent experimental results that compare the performances of LT against conventional diffraction tomography (DT) based on Born and Rytov approximations.

The experiments were carried out on a three-dimensional arrangement of yeast cells dispersed in an agarose gel. The cells form clusters of different sizes and thus allows us to assess the effect of sample complexity on the various algorithms. We used an interferometric imaging system to measure the complex optical field scattered from the sample, which was illuminated with plane wave at different incidence angles.

2.

METHOD

In this section, we start by stating the problem of inverse scattering involved in optical tomography. We then provide the detail of DT and LT algorithm, as well as the iterative version of the DT.

2.1

The inverse scattering problem

The inverse scattering problem consists in retrieving the characteristics of an unknown object by probing it with known incident waves and measuring the scattered waves. In this paper, we consider a transparent object of unknown refractive index n(r) (r = (x, y, z)). The object is described by its the scattering potential defined as F(r) = k2(n(r)2n02)/4π, with n0 the background refractive index and k the wave vector of the incident light. F is the function that we aim at retrieving. The object is illuminated with a sequence of plane waves at different angles. For every incidence angle, the total field (incident and scattered) is recorded using digital holography, so that we gain knowledge of the amplitude and phase of the field. The field us scattered by the object can be calculated using the Lippmann-Schwinger equation:

00388_psisdg10333_1033306_page_2_1.jpg

where u is the total optical field, ui the incident field.

2.2

Linear scattering

Equation (1) can be linearized if the scattered field us is much smaller than the incident field ui. In this case, u can be replaced by ui in the integrand, which is referred to as the first Born approximation, and we get:

00388_psisdg10333_1033306_page_2_2.jpg

The assumption that the scattered field is much smaller that the incident field implies that the field resulting from the re-scattering of us will be totally negligible. The first Born approximation is thus equivalent to the assumption of single scattering. With the linearized equation (2), it is possible to solve the inverse scattering problem (i.e. retrieve F from the collection of measurements of u) using the direct inversion formula proposed by E. Wolf [8], which we shall hence call the Wolf transform:

00388_psisdg10333_1033306_page_2_3.jpg

where f is the scattering amplitude, k the wave vector in the direction of observation and ki the wave vector of the incident field. 3D denotes the three-dimensional Fourier transform of the scattering potential. The scattering amplitude is related to the scattered field by usB(r) = f(k) exp(jk|r|)/|r|, for |r| sufficiently large. Note that equation (3) is valid for plane wave illumination. For more complex incident field, we have to linearly add the scattering amplitude corresponding to each Fourier mode of the incident field.

As we will see in the results section, the Wolf transform produce reconstructions that are strongly distorted, typically close to the edges for spheroidal objects like beads or biological cells. The above-mentioned distortions are suppressed efficiently by using the Rytov approximation, which is an alternate way of linearizing equation (1) that is widely used in optical tomography and was initially proposed by Devaney in that context [13]. The underlying idea is to express the field u in term of a complex phase ψ: u = ejkψ, which is nothing more than a change of variable. We further define ui = ejkψi and δψ = ψψi so that we have u = ui ejkδψ. When inserted into Helmholtz equation, this lead to a Riccati equation in term of δψ. The Ryotv approximation consists in retaining only the first term of the Neumann expansion of the Riccati equation [13], which results in a linear equation that happens to have the same form as Helmholtz equation. For that reason, the Wolf transform, which normally applies to the field u, can now be applied to δψ directly. We can write:

00388_psisdg10333_1033306_page_2_5.jpg

For the Born approximation, we take the digital two-dimensional Fourier transform of the field u measured in the plane of the detector, which is related to f by the following equation:

00388_psisdg10333_1033306_page_2_6.jpg

where the tilde denotes the two-dimensional Fourier transform and fB the scattering amplitude under Born approximation. Note that, this formula is valid for a plane wave incident field: ui = exp(jki·r). For Rytov approximation, we make use of equation (4):

00388_psisdg10333_1033306_page_2_7.jpg

where fR is the scattering amplitude under Rytov approximation and 𝓕 the Fourier transform. At this point, the procedure is the same for both approximations, fB or fR is used to populate the three-dimensional Fourier space of the object according to equation (3).

For each incident plane wave (indexed by p up to the total number of views N), the scattering amplitude fp is used to populate the Fourier domain of the object function according to the Wolf Transform (3):

00388_psisdg10333_1033306_page_3_1.jpg

The scattering potential is obtained by taking the inverse three-dimensional Fourier transform of the previous equation:

00388_psisdg10333_1033306_page_3_2.jpg

where Δz is the discretization step in direction z. Finally, the refractive index distribution reconstructions is obtained from F:

00388_psisdg10333_1033306_page_3_3.jpg

according to Born and Rytov approximations respectively.

2.3

Learning tomography

We briefly review here the basic concepts used in LT. A digital map n(x, y, z) of the refractive index (the estimate) of the unknown object is generated in the computer and a forward model H simulating the physics of light propagation through the sample is used to predict the set of measurements i that would be obtained from the current estimate and a set of incident illumination waves uinc, i. This prediction is compared to the actual measurement mi performed experimentally on the real sample. The goal of LT is to ‘learn’ the digital map of the refractive index that minimizes the residual r between the actual measurement and the prediction, i.e. find the minimum of the following cost function C:

00388_psisdg10333_1033306_page_3_4.jpg

where D is the data fidelity term and R contains additional constraints such as a regularizing operator. The parameter τ determines the relative weight given to the additional constraints. The estimate of the refractive index will then be given by:

00388_psisdg10333_1033306_page_3_5.jpg

The operator H that maps the refractive index distribution to the measured field is nonlinear. This can be understood from Eq. (1) in which the scattered field is a nonlinear function of the scattering potential since doubling the strength of the scattering potential does not double the strength of the scattered field. Due to the nonlinearity, the cost function is generally not convex. Additionally, the inverse problem of recovering the refractive index from the light measurements is ill posed, which is addressed by regularizing the solution with the isotropic total-variation (TV) of the refractive index contrast and positivity constraints:

00388_psisdg10333_1033306_page_3_6.jpg

where Dj represent the discrete variation in direction j. The TV regularization favors piecewise constant solutions. Indeed, we expect biological tissues to consist mainly of constant refractive index parts separated by sharp boundaries, such as organelles in a cell.

In order to solve the problem posed in Eq. (11), we use the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) [14]. The FISTA does not require computing the gradient of the total cost function C, but only the gradient of the data fidelity term D. The regularization is taken care of by solving the following sub-problem at each iteration k:

00388_psisdg10333_1033306_page_4_1.jpg

We use the Spilt-Step Fourier (SSF) Beam Propagation Method (BPM) [11, 12] as a forward model because the gradient of the data term can then be computed efficiently. At each iteration, we randomly select a subset of Neff (typically 8) views from the set of all the views. We compute the gradient of the data fidelity term D for the mj in that subset of views with respect to n and we update the refractive index map accordingly:

00388_psisdg10333_1033306_page_4_2.jpg

where nk is the refractive index estimate at iteration k and s the step size. In addition to the TV regularization, we impose a positivity constraint on the refractive index contrast, i.e. we set n ← max(n, n0) at each iteration, where n0 is the background refractive index.

The initial guess, i.e. nk=0, is taken to be constant and equal to n0 for weakly scattering objects. For high index contrast, the nonlinearity of H is stronger and the cost function generally has many local minima. In order to converge. Therefore is desirable that we start with an initial guess that is close enough to the solution to avoid the LT algorithm to fall in such minima. The generation of the initial guess is discussed in the next section.

2.4

Linear iterative model

A fair comparison between linear and nonlinear models can only stand if both are subjects to the same type of regularization and positivity constraints. In order to produce such an iterative linear algorithm, we simply take the LT algorithm in which we replace the nonlinear forward model, i.e. H in Eq. (10), by a linear model A. Specifically, we replace the BPM, which is nonlinear, by the Wolf transform, which is linear under the Born or Rytov approximations. These steps are detailed below. Let’s define the indicator function M of the set of tomographic measurements in the Fourier space:

00388_psisdg10333_1033306_page_4_3.jpg

where kd is the detection direction vector, ki the incident wave vector, and K the coordinate in the (three-dimensional) Fourier domain of the sample. For convenience, the sample is described here by its scattering potential function F. According to the Wolf transform, all the tomographic measurements lie on the locus of point where M = 1, i.e. the union of all the subset of the Ewald spheres that correspond to the measurements mi. In that context, we refer to the measurement m as being the collection of the projections of mi(kd, ki) onto the Ewald spheres, i.e. mi(K). Note that some measurements will fall on the same location K (e.g. if kd1-ki1= kd2-ki2), in this case, we simply average all the contributions at the same location. The linear forward operator A is a three-dimensional Fourier transform of the scattering potential F followed by a masking operation using the function M:

00388_psisdg10333_1033306_page_4_4.jpg

The cost function is now defined as:

00388_psisdg10333_1033306_page_4_5.jpg

The gradient of the data term D is then:

00388_psisdg10333_1033306_page_4_6.jpg

where r is the residual now only evaluated where M = 1. From this point, the procedure is exactly the same as for the LT algorithm. We then use equations (13) and (14) as in LT in order to enforce the TV and positivity constraints.

2.5

Experimental method

Experimental apparatus

The experimental apparatus is shown in Fig. 1. It consists in a Mach-Zehnder interferometer. The sample is mounted between two 150 micron thick cover slips and sits between two Olympus oil immersion PlanApo 100x NA1.4 objectives in the signal arm of the interferometer. The light source is a continuous wave diode laser at 406nm, with a coherence length of approximately 100 microns. The beam is spatially filtered, expanded and focused in the back focal plane of the illumination objective (IO) thus projecting a plane wave on the sample. A couple of galvo-mirrors are placed before the illumination objective, one steering the beam in the x dimension and one in the y dimension. The surfaces of the galvo-mirror lie in conjugate image planes and are imaged in turn onto the sample. By changing the angles of the galvo- mirrors, we can change the incidence angle of the plane wave illuminating the sample. The sample is imaged with a lateral magnification of 111 through the collection objective (CO) and a relay lens on a detector array (Andor Neo EMCCD, pixel size 6.5 microns), where a hologram is formed through interference with the reference beam. For each view, we measure the total field u with the object in place. We then move the sample together with the cover slips to the side so that the field of view becomes clear of any object. We record then a second field, which is in fact a measurement of the incident field ui itself. This technique allows us to compensate aberration present in the optical system.

Fig. 1:

Experimental apparatus. The light source is a continuous-wave laser diode at 406nm. The beam is spatially filtered, expanded and split into a reference wave and a signal wave. The signal wave incidence angle can be steered with two galvo-mirrors. The sample is placed between two 100x microscope objectives and imaged on a CCD camera, where a digital hologram is captured by recombining the reference beam.

00388_psisdg10333_1033306_page_5_1.jpg

Sample preparation

The samples used in the experiments presented in this article are live yeast cells dispersed in a three-dimensional agarose gel. The yeast cells are diluted in a phosphate buffer saline (PBS) solution at the desired concentration at room temperature. A water solution containing 0.7% agarose is heated up to 80°C and then left to cool down to 40°C at which point a drop of the PBS solution containing the yeast cells is added. The resulting solution is placed between cover slips and left to cool down to room temperature where it becomes solid. This results in a three-dimensional arrangement of cells. The distance between the cells and thus the amount of overlap can be adjusted by varying the initial concentration.

3.

RESULTS

The linear DT reconstruction was performed according to the Born [8] and Rytov approximation [10] from a collection of measurements taken at 160 illuminations angles arranged in a circle in k-space. For the linear DT, we also used the iterative algorithm described in section 2.4 above. The LT reconstruction was performed from the same measurements, but randomly selecting only eight views at every iteration of the LT algorithm. The random selection of small number of views has the advantage of significantly accelerating the computation without impairing the convergence characteristics. The selection of random views is, in general, is helpful to prevent the algorithm being stuck in. For the LT shown in Fig. 2, there were 50 iterations in total. In order to converge, the LT algorithm needs to be regularized. We globally impose a total variation (TV) regularization that forces solutions to be piecewise constant. We also enforce the non-negativity of the refractive index contrast, the cells having a larger index than the surrounding medium (essentially water). In order to display a fair comparison between LT and DT, we applied the same regularization and positivity constraint to the DT reconstruction. For that, we devised an iterative linear algorithm that is similar to LT except that the nonlinear propagation (the BPM) is replaced by the linear Wolf transform. The results are shown in Fig.2.

Fig. 2:

The images show an x-y slice from the three-dimensional reconstruction of the refractive index of a group of yeast cells obtained with different methods: (a) under the Born approximation, (b) under the Rytov approximation and positivity constraint applied once at the end, (c) under iterative Rytov approximation with TV regularization and positivity constraint, (d) using LT with TV regularization and positivity constraint. The gray scale linearly maps to the refractive index n.

00388_psisdg10333_1033306_page_6_1.jpg

In Fig. 2(a), we can see that the reconstruction using the diffraction tomography algorithm based on the Born approximation suffers from severe loss of contrast. This is an inherent limitation of the Wolf formula using the Born approximation. Any object-induced phase shift larger than π/2 will lead to a partially inverted contrast index map. The Born approximation, is thus limited to objects that induce less than a π/2 phase shift. Individual yeast cells display a phase shift around 2π and will thus appear with a dark body (underestimated refractive index). However, the edges are partially preserved as they induce strong scattering and ripples in the phase. If the contrast is locally inverted, the edge will appear slightly displaced but will still constitute a prominent feature, which the reason why the outlines of the cells are visible in the Born reconstruction. For the Rytov reconstruction, the phases of the measurements have been unwrapped using a two-dimensional phase unwrapping algorithm (PUMA) [15]. Even though the Rytov approximation based DT makes use of the Wolf transform (exactly as Born-approximation based DT), the unwrapping of the measurement phase proves to be crucial as it restores the proper contrast in the reconstruction. The same is true for the iterative DT based on Rytov approximation as the same unwrapped measurement are used (Fig. 2, bottom left). The improvement by using the iterative linear algorithm consists in enforcing a piecewise constant refractive index distribution, which can be seen by comparing the iterative and simple Rytov reconsturctions. LT provides even better performance in term of image quality (Fig. 2, bottom right). We attribute this to be the combined effect of regularization and accounting for multiple scattering.

4

CONCLUSION

In this paper, we experimentally compared the performance of the DT algorithm based on a linear forward model, i.e. the Wolf Transform, and the LT algorithm based on beam propagation, which is a nonlinear model. The Born approximation is clearly too weak to yield index maps with proper contrast for typical yeast cells. The Rytov approximation together with phase unwrapping of the measurement performs much better. In this case, as long as the phase can be unwrapped, the proper index contrast is obtained in the reconstruction. The Rytov based DT can be further improved through the use of regularization. In this case, the background noise is reduced and the contrast is enhanced. LT provides the best results in term of contrast and background suppression.

REFERENCES

[1] 

V. Lauer, “New approach to optical diffraction tomography yielding a vector equation of diffraction tomography and a novel tomographic microscope,” Jour. of Mic., 205 165 –176 (2002). Google Scholar

[2] 

F. Charrière, A. Marian, F. Montfort, J. Kuehn, T. Colomb, E. Cuche, P. Marquet, and C. Depeursinge, “Cell refractive index tomography by digital holographic microscopy,” Opt. Lett., 31 (2), 178 –180 (2006). https://doi.org/10.1364/OL.31.000178 Google Scholar

[3] 

W. Choi, C. Fang-Yen, K. Badizadegan, S. Oh, N. Lue, R. R. Dasari, and M. S. Feld, “Tomographic phase microscopy,” Nat. Meth., 4 717 –719 (2007). https://doi.org/10.1038/nmeth1078 Google Scholar

[4] 

W. Choi, C. Fang-Yen, K. Badizadegan, R. R. Dasari, and M. S. Feld, “Extended depth of focus in tomographic phase microscopy using a propagation algorithm,” Opt. Lett., 33 (2), 171 –173 (2008). https://doi.org/10.1364/OL.33.000171 Google Scholar

[5] 

Y. Sung, W. Choi, Christopher Fang-Yen, K. Badizadegan, R. R. Dasari, and M. S. Feld, “Optical diffraction tomography for high resolution live cell imaging,” Opt. Exp., 17 (1), 266 –277 (2009). https://doi.org/10.1364/OE.17.000266 Google Scholar

[6] 

Y. Cotte, F. Toy, P. Jourdain, N. Pavillon, D. Boss, P. Magistretti, P. Marquet, and C. Depeursinge, “Marker-free phase nanoscopy,” Nat. Phot., 7 (2), 113 –117 (2013). https://doi.org/10.1038/nphoton.2012.329 Google Scholar

[7] 

L. Tian and L. Waller, “3D intensity and phase imaging from light field measurements in an LED array microscope,” Optica, 2 (2), 104 –111 (2015). https://doi.org/10.1364/OPTICA.2.000104 Google Scholar

[8] 

E. Wolf, “Three-dimensional structure determination of semi-transparent objects from holographic data,” Opt. Comm., 1 (4), 153 –156 (1969). https://doi.org/10.1016/0030-4018(69)90052-2 Google Scholar

[9] 

V. I. Tatarski, Wave Propagation in a Turbulent Medium, McGraw-Hill,1961). Google Scholar

[10] 

A. J. Devaney, “Inverse-scattering theory within the Rytov approximation,” Opt. Lett., 6 (8), 374 –376 (1981). https://doi.org/10.1364/OL.6.000374 Google Scholar

[11] 

U. S. Kamilov, I. N. Papadopoulos, M. H. Shoreh, A. Goy, C. Vonesch, M. Unser, and D. Psaltis, “Learning approach to optical tomography,” Optica, 2 (6), 517 –522 (2015). https://doi.org/10.1364/OPTICA.2.000517 Google Scholar

[12] 

U. S. Kamilov, I. N. Papadopoulos, M. H. Shoreh, A. Goy, C. Vonesch, M. Unser, and D. Psaltis, “Optical tomographic image reconstruction based on beam propagation and sparse regularization,” IEEE Trans. on Comp. Im., 2 (1), 59 –70 (2016). Google Scholar

[13] 

Anthony J. Devaney, Mathematical Foundations of Imaging, Tomography and Wavefield Inversion, Cambridge University Press,2012). https://doi.org/10.1017/CBO9781139047838 Google Scholar

[14] 

A. Beck and M. Teboulle, “Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems,” in IEEE Trans. On Im. Proc., 2419 –2434 (2009). Google Scholar

[15] 

J. M. Bioucas-Dias and G. Valadão, “Phase Unwrapping via Graph Cuts,” 16 (3), (2007). Google Scholar
© (2017) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
M. Hasani Shoreh, A. Goy, J. Lim, U. Kamilov, M. Unser, and D. Psaltis "Imaging cell clusters and tissue using learning tomography", Proc. SPIE 10333, Optical Methods for Inspection, Characterization, and Imaging of Biomaterials III, 1033306 (26 June 2017); https://doi.org/10.1117/12.2275250
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Refractive index

Scattering

Tomography

Optical tomography

Beam propagation method

Reconstruction algorithms

Yeast

Back to Top