We propose to use deep convolutional neural networks (DCNNs) to perform 2D and 3D computational imaging. Specifically, we investigate three different applications. We first try to solve the 3D inverse scattering problem based on learning a huge number of training target and speckle pairs. We also demonstrate a new DCNN architecture to perform Fourier ptychographic Microscopy (FPM) reconstruction, which achieves high-resolution phase recovery with considerably less data than standard FPM. Finally, we employ DCNN models that can predict focused 2D fluorescent microscopic images from blurred images captured at overfocused or underfocused planes.
We propose to use deep convolutional neural networks (DCNNs) to perform 2D and 3D imaging through scattering media. The inverse scattering problem is solved based on learning a huge number of training target and speckle pairs. The proposed technique does not rely on a reference beam, thus employs a simpler optical setup than previous techniques without the need to know the imaging model and optical processes. This lack of the need to know a prior model of the forward operator is very important since many optimization techniques are very sensitive to errors caused by the inaccuracy of the forward model.
Deep convolutional neural networks (DCNNs) offer a promising performance for many image processing areas, such as super-resolution, deconvolution, image classification, denoising, and segmentation, with outstanding results. Here, we develop for the first time, to our knowledge, a method to perform 3-D computational optical tomography using 3-D DCNN. A simulated 3-D phantom dataset was first constructed and converted to a dataset of phase objects imaged on a spatial light modulator. For each phase image in the dataset, the corresponding diffracted intensity image was experimentally recorded on a CCD. We then experimentally demonstrate the ability of the developed 3-D DCNN algorithm to solve the inverse problem by reconstructing the 3-D index of refraction distributions of test phantoms from the dataset from their corresponding diffraction patterns.
Digital holographic microscopy (DHM) provides label-free and real-time quantitative phase information relevant to the analysis of dynamic biological systems. A DHM based on telecentric configuration optically mitigates phase aberrations due to the microscope objective and linear high frequency fringes due to the reference beam thus minimizing digital aberration correction needed for distortion free 3D reconstruction. The purpose of this work is to quantitatively assess growth and migratory behavior of invasive cancer cells using a telecentric DHM system. Together, the height and lateral shape features of individual cells, determined from time-lapse series of phase reconstructions, should reveal aspects of cell migration, cell-matrix adhesion, and cell cycle phase transitions. To test this, MDA-MB-231 breast cancer cells were cultured on collagen-coated or un-coated glass, and 3D holograms were reconstructed over 2 hours. Cells on collagencoated glass had an average 14% larger spread area than cells on uncoated glass (n=18-22 cells/group). The spread area of cells on uncoated glass were 15-21% larger than cells seeded on collagen hydrogels (n=18-22 cells/group). Premitotic cell rounding was observed with average phase height increasing 57% over 10 minutes. Following cell division phase height decreased linearly (R<sup>2</sup>=0.94) to 58% of the original height pre-division. Phase objects consistent with lamellipodia were apparent from the reconstructions at the leading edge of migrating cells. These data demonstrate the ability to track quantitative phase parameters and relate them to cell morphology during cell migration and division on adherent substrates, using telecentric DHM. The technique enables future studies of cell-matrix interactions relevant to cancer.
In this paper, we present detail analysis and a step-by-step implementation of an optimized fringe projection profilometry (FPP) based 3D shape measurement system. First, we propose a multi-frequency and multi-phase shifting sinusoidal fringe pattern reconstruction approach to increase accuracy and sensitivity of the system. Second, phase error compensation caused by the nonlinear transfer function of the projector and camera is performed through polynomial approximation. Third, phase unwrapping is performed using spatial and temporal techniques and the tradeoff between processing speed and high accuracy is discussed in details. Fourth, generalized camera and system calibration are developed for phase to real world coordinate transformation. The calibration coefficients are estimated accurately using a reference plane and several gauge blocks with precisely known heights and by employing a nonlinear least square fitting method. Fifth, a texture will be attached to the height profile by registering a 2D real photo to the 3D height map. The last step is to perform 3D image fusion and registration using an iterative closest point (ICP) algorithm for a full field of view reconstruction. The system is experimentally constructed using compact, portable, and low cost off-the-shelf components. A MATLAB® based GUI is developed to control and synchronize the whole system.
Interferometric based techniques are often used for 3D quantitative phase imaging. While these techniques are sensitive to vibrations, non-interferometric intensity based techniques such as the transport of intensity equation (TIE) do not suffer from such a drawback. Phase reconstruction of phase objects using TIE technique is accomplished by recording several diffraction patterns at different observation planes through axially translating the CCD. In this paper, we purpose to use a spatial light modulator (SLM) in a modified 4f TIE optical setup to acquire 3D tomographic images of phase objects. This modified setup will reduce the acquisition time dramatically making the TIE technique useful for dynamic events such as biological samples. We illustrate how 3D phase objects can be reconstructed tomographically by constructing a rotating mechanism for the sample. At each angle of rotation, two diffraction patterns are captured by the CCD either sequentially or instantaneously with the help of a reference mirror. The reconstructed optical fields are tomographically recomposed to yield the final 3D shape using a tomographic backprojection technique. Finally, a reconfigurable hardware controlled by a GUI is employed to synchronize the CCD, the SLM and the rotating stage.
In this work we will extend the traditional TIE setup of phase retrieval of a phase object through axial translation of the CCD by employing a tunable lens (TL-TIE). This setup is also extended to a 360° tomographic 3D reconstruction through multiple illuminations from different angles by rotating the phase object. Finally, synchronization between the CCD, and the tunable lens is employed using a reconfigurable hardware to automate the 3D 360° tomographic reconstruction process.