Nowadays, three-dimensional (3-D) profiling and tracking microscale objects have been receiving much attention due to their wide applications. For example, microsphere or microbubble counting and locating in a 3-D space occur in the fields of microfluidics, suspension rheology, crystallization, etc.22.214.171.124.126.96.36.199.10.–11 Many microorganisms swim in 3-D and helical paths. Thus, measuring their 3-D trajectories in time is essential to obtain detailed information on biophysical processes, such as motile behavior and dynamic performance.1213.–14 Quantitative analyses of cancer cell locomotion and shape change in a 3-D environment reveal the biological characteristics for clinical need.15188.8.131.52.–20 3-D imaging of randomly oriented microfibers and their interactions with surrounding free-swimming cells opens up new perspectives in cases where quantitative measurement of characteristics (size, length, orientation, speed, displacement, etc.) is of great interest.21
Various approaches have been demonstrated for 3-D profiling and tracking of micro-sized objects, and optical techniques and numerical localization algorithms have been widely chosen as remarkable tools since they have the advantages of being full-field, label-free, noncontact, noninvasive, etc.2184.108.40.206.220.127.116.11.18.104.22.168.22.214.171.124.126.96.36.199.–43 Digital holographic microscopy (DHM) is an emerging technology of a new paradigm in general imaging and biomedical applications.44,45 Various techniques of DHM, including off-axis DHM, in-line DHM, quantitative phase microscopy by digital holography (DH-QPM), etc., have been proven to be potent tools to profile and track micro-sized objects in 3-D volumes. In conventional microscopy techniques, only two-dimensional focused images on a fixed plane can be recorded, while information not in the focal plane would be permanently missed. However, DHM provides a tool to refocus an object and is able to record a hologram containing all the real-time, 3-D structures of an object in the absence of mechanical focus adjustment. And the information is available in digital form for postprocessing. DHM has been shown to be the key to 3-D particle image velocimetry (PIV)4647.48.49.–50 since the conventional PIV techniques have the inherent limitations of thin depth of field, instrument complexity, and impossibility of real-time imaging.5152.–53 However, DHM is able to overcome these and has the simplicity of the microscope and sample preparation, maximum information and resolution, reduction in time and data amount, etc. Aberrations and background distortions of the optical field can be minimized by available DHM techniques. The DHM for 3-D profiling and tracking generally is a two-step process: first a hologram is recorded digitally, and then the hologram is numerically reconstructed in different depths to yield an image of the object by various numerical diffraction methods, such as angular spectrum method,44,45 Kirchhoff-Helmholtz transform,54 Fresnel transform,55,56 Huygens convolution,44,45 etc. This reconstruction results in complex field of object, and one can extract its amplitude or phase profile to represent the object. Investigations of the third dimension based on the reconstructed data are then performed to determine focal planes and complete the 3-D pattern recognition. For a moving object, the reconstruction of successive holographic data yields a complete four-dimensional space-time record of the dynamic process. DHM is demonstrated to have the capacity of monitoring the 3-D distribution and motion pattern of particles, living cells, and fibers with different shapes (spherical, needle shaped, and randomly oriented), size (few to hundred micrometer), and conditions (static, suspended, and flow-through) in real time.
Description of several representative DHM techniques is introduced in Sec. 2, and the holographic information processing methods for 3-D profiling and tracking are given in Sec. 3. Wide and active field of applications on 3-D profiling and tracking of different samples, including microparticles, bubbles, biological cells, microfibers, etc., by DHM techniques are introduced in Sec. 4, and the conclusion is provided in Sec. 5. This review has an emphasis on applications of DHM in the field of 3-D profiling and tracking and, therefore, omits some major areas, such as digital holography principle and development, theoretical studies of digital holography, special digital holography techniques, etc.
A basic DHM setup consists of an illumination source, an interferometer, a digitizing camera, and a computer with necessary programs. Most often a laser is used for illumination with the necessary coherence to produce interference. For multiwavelength techniques, two or more different lasers can be coupled into the interferometer, or a tunable laser can be employed. There are also low-coherence techniques for the purpose of reducing speckle and spurious interference noise or generating contour or tomographic images. Even an LED typically has 30 μm or so coherence length, which can be sufficient for holographic microscopy.
In this review, we refer to Gabor holography as in-line holography.57 In-line DHM is a type of microscopy without objective lenses, and as illustrated in Fig. 1, a single light beam directed onto a pinhole of a diameter of the order of a wavelength illuminates the object, typically several thousand wavelengths away from the pinhole, and the object beam is the part of the incident light that is scattered by the object and the unscattered remainder is taken as the reference beam. If the light source is coherent, the pinhole is not necessary. The object field is automatically in alignment with the reference beam, and the interference of the object and reference beams results in the holographic diffraction pattern, which is recorded by CCD camera and then transferred to a computer for numerical reconstruction.
By removing background effects first and then reconstructing the object field at different planes, a 3-D image can be built up from a single two-dimensional image, and the entire hologram pixel count is utilized, which also leads to shorter minimum distances for reconstruction and higher resolution of the resultant image. With the simplicity of the apparatus and large depth of field, the in-line DHM is particularly useful for particle image analysis. In conjunction with reconstruction algorithm, the 3-D position of a particle can be determined and then applied to particle velocimetry, such as tracking small particles or swimming cells in a liquid flow.
In an off-axis DHM setup, a Mach-Zehnder interferometer is chosen since it offers flexibility in alignment, especially when microscopic imaging optics is used. One microscope objective lens (MO) is needed for object magnification and another one is used in the reference arm to match the curvatures of the object and reference wave fronts. We illustrated the off-axis holographic microscopy setup in Fig. 2. The object arm contains a sample stage and an MO that projects a magnified image of the object onto a CCD camera. The reference arm similarly contains another MO, and the reference and object waves are offset by an angle to avoid the overlap of the reference and the twin images, so that the holographic interference pattern contains fringes due to interference between the diffracted object field and the off-axis reference field. The captured holographic image is numerically converted into Fourier domain to obtain the angular spectrum,44,45 and a spatial filter is then applied to retain the real image peak alone. The filtered angular spectrum is propagated to appropriate distance and, by an inverse Fourier transform, reconstructed as an array of complex numbers containing the amplitude and phase images of the sample. Both amplitude and phase profiles can be further analyzed to quantitatively determine the positions of objects in a 3-D space.
Quantitative Phase Microscopy by Digital Holography
Off-axis DHM is a very effective process for achieving high-precision QPM since it allows measurement of optical thickness with nanometer-scale accuracy by single-shot, wide-field acquisition, and it yields phase profiles without some of the complications of other phase imaging methods. The phase image is immediately and directly available on calculating the two-dimensional complex array of the holographic image, and the phase profile conveys quantitative information about the physical thickness and index of refraction of cells. We have recently utilized DH-QPM to study the wrinkling of a silicone rubber film by motile fibroblasts.58 The wrinkle formation has been visualized, quantitative measures of surface deformation have been extracted, and cellular traction force has been estimated in a direct and straightforward manner. A nonwrinkling substrate, collagen-coated polyacrylamide (PAA) was also employed to make direct measurement of elastic deformations, albeit at discrete locations.59 The Young’s modulus of PAA can be adjusted by controlling the concentrations of the monomer and cross-linker. DH-QPM, with its capacity of yielding quantitative measures of deformation directly, has been employed to measure the Young’s modulus of PAA,60 which provided a very effective process for achieving high-precision quantitative phase microscopy. We have utilized DH-QPM to study motile fibroblasts deforming PAA gel,61 where the cell-substrate adhesion has been visualized and quantitative measures of surface deformation have been extracted. The substrate stiffness and quantitative measures of substrate deformation have been combined to produce estimates of the traction forces and characterize how these forces vary depending on the substrate rigidity.
The twin-beams DHM setup is illustrated in Fig. 3.62 Two beams, coming from the same laser source and slightly off-axis, enter into the microscope objective and result in an image on a CCD plane. Theoretically, the two beams have the same focal plane; however, the twin beams can experience aberrations resulting in some focus-shift. As shown in Fig. 3, each particle of the sample appears as two projections on the CCD array. The tracking is performed by evaluating the two out-of-focus projections of the particles due to the twin-beams onto the CCD plane, and the separation between the two projections is a function of the longitudinal position of the particle. QPM images of the micro-objects can be directly obtained from the holograms recorded by the same configuration.
Numerical Focus Methods for Three-Dimensional Profiling and Tracking by DHM
Quantification of Image Sharpness and Peak Searching
To localize an object in a 3-D volume, an automated data analysis algorithm to accurately identify depth position is required. For example, pure amplitude objects located out of focal plane appear as gray patches without sharp structures, while pure phase objects show minimum visibility when they are in focus.6364.–65 Thus, focusing metrics based on the quantification of image sharpness are applied by searching on the reconstructed images for the planes that contain the objects with the peak and sharpest details to establish the all-in-focus profile and its corresponding depth map.66 At each pixel, the intensity variation in longitudinal direction is investigated to find out the peak intensity. The peak intensity value becomes a pixel of an all-in-focus profile and its location in longitudinal direction is recorded as a corresponding depth map value. The combination of resultant all-in-focus profile and depth map enables one to produce the 3-D visualization of objects, and object properties can then be extracted.
We recently utilized off-axis DHM and numerical autofocusing algorithm based on peak searching to image suspended polymer microspheres (9.6 μm in diameter) in suspension.21 Figure 4(a) presents the hologram of microspheres in suspension captured by off-axis DHM. Angular spectrum method44,45 was applied and the reconstructed amplitude image was shown in Fig. 4(b), where several in-focus and out-of-focus microspheres were seen at this reconstruction plane. The field of view was with . The microsphere focused the incoming light and formed a bright spot along the optical path. Therefore, the pattern of the microsphere appeared to have a maximum intensity at the center near the in-focus plane and the surrounding was dark. Then, the autofocusing algorithm based on peak searching was applied to identify the in-focus position of each particle. The hologram was numerically reconstructed in longitudinal direction from 100 to , in steps. The autofocusing algorithm was then applied by searching on the reconstructed images for the planes that contained the objects with the peak and sharpest details to establish the all-in-focus profile. The peak intensity at each pixel along the longitudinal direction was chosen and this value became a pixel of an in-focus profile; meanwhile, its location in longitudinal direction was also recorded as a corresponding depth map value. The combination of all-in-focus intensity profiles [Fig. 4(c)] and the corresponding depth maps [Fig. 4(d)] enabled to produce the 3-D visualization of microspheres and a threshold on the intensity allowed for distinguishing objects from other elements, which is illustrated in Fig. 4(e). In fact, we took every center position of brightest points in Fig. 4(c) as locations of particles, and Fig. 4(d) was used to determine the -location of those particles identified in Fig. 4(c). Figure 4(d) is mostly noisy and information of only the locations where the particles were present was actually used. Figures 4(f) to 4(h) show the , , and views of the 3-D profile, respectively. This straightforward autofocusing algorithm is able to determine focal planes for all the objects in the reconstructed volume automatically. The combination of depth map and axial position of objects allows for quantitative 3-D profiling. This method has been widely used by researchers, and more applications will be discussed in Sec. 4.
3-D Deconvolution Methods
Overviews of the deconvolution methods regarding the out-of-focus problem in optical microscopy can be found in Refs. 6768.69. to 70. In Ref. 71, 3-D deconvolution methods, including instant and iterative deconvolution, have been applied to restore particle distribution from holograms and the corresponding reconstructions. Instant 3-D deconvolution could be applied on separated particles spread in a certain volume and allowed for the identification of the depth position of object and removal of out-of-focus information. The way to perform the instant 3-D deconvolution was to apply it to reconstructed intensity distributions from holograms using Eq. (1):Fig. 5(a). The reconstructions were performed at continuous planes in the volume and the PSF was chosen as the reconstruction of the hologram of a point scatter located at a fixed plane. The results of 3-D deconvolution from Eq. (1) showed a sharp 3-D visualization of individual scatters constituting the three letters, which were seen to be distinguished from each other, as shown in Fig. 5(b).
Iterative deconvolution could be applied on objects of continuous and extended shapes and distributions. The way to perform the iterative deconvolution was to apply it to reconstructed complex fields from holograms. The iterative loop includedFig. 5(c) was placed at three different planes. With the same parameters of reconstruction as the instant 3-D deconvolution case, the iterative deconvolution was performed by the iterative loop described above in Eq. (2). The result showed the three continuous letters, which were seen to be clearly distinguished in a 3-D volume [Fig. 5(d)]. These two methods were also demonstrated by experimental in-line holograms of polystyrene microspheres randomly locating on both sides of a thin glass. A cut-out from the original hologram of microspheres, which included only a single sphere, was taken as the PSF and the 3-D localization of microspheres was retrieved. Moreover, a simulated PSF was also tested and a comparable good result was obtained.
Rayleigh-Sommerfeld Backpropagation Method
Rayleigh-Sommerfeld backpropagation method is a general and fast volume reconstruction analysis tool for DHM.55,72,73 A well-known Gouy phase anomaly55,74 can be used to discriminate between objects lying on either side of the holographic image plane. A recorded in-line hologram was reconstructed using Rayleigh-Sommerfeld diffraction integral to refocus the scattered object at a particular position and observe the phase anomaly as a contrast inversion about the object’s position by Wilson et.al.72 If the complex field of scattering objects was obtained in the focal plane, it could be reconstructed at plane above the focal plane as a convolution of amplitude in the focal plane with Rayleigh-Sommerfeld propagator, as in Eq. (3):5572 and if a particle is located in positive -direction, the field was converging when it arrived at the image sensor and had not yet passed through the geometrical focus. This allowed for determining the location of the center of the object and separating objects on different sides of the hologram plane. Particles always had twin images in the reconstructed space no matter they are located above or below the focal plane. If only the upstream of the focal plane was reconstructed, the intensity gradient was a maximum for particles with and a minimum for those with . A micro-sized diameter sphere was tested by this technique and a typical volumetric reconstructed rays converging to the bright point above the focal plane was shown in Fig. 6. Another analysis of an experimental sample consisted of closely spaced particles on a glass slide showing the applicability of this method to distinguish weakly scattering particles, such as colloids or planktons locating at close lateral but separate axial positions.
Compressive Holographic Method
Compressive sensing is a technique to efficiently acquire the information of signal in the underdetermined linear system supported by the sparseness of the signal.75 Hence, the higher dimensional signal can be recovered by the lower-dimensional measurements. Recently, some research groups have been studying about the compressive holography, which applied the concept of compressive sensing to holography.7576.77.–78 In holographic recording, 3-D data of the object were recorded as a two-dimensional complex hologram. Supported by the condition that the distribution of the object in 3-D space was sufficiently sparse, the compressive sensing technique could retrieve the 3-D distribution from two-dimensional complex hologram. Brady et al.75,76 had successfully shown the possibility of the compressive holography with the hologram recorded by in-line DHM. In-line DHM, because of its simplicity and efficiency, provided a potent tool for compressive sampling by recording a 3-D object onto a two-dimensional focal plan array, which was related to Fourier transform of the object field. Compressive holography collected fewer amounts of measurements than voxels in the reconstructions. The 3-D datacube was encoded and compressed into two-dimensional holographic measurement by holographic sampling process, and the encoding was then inverted using the compressive sampling theory.
Two seed parachutes were illuminated and placed away from the detector array at different distance separately [Fig. 7]. After reconstruction, the stem and petals representing the high-frequency features in the image were clearly shown and the distance between the two parachutes were also estimated. The final result presented a 3-D datacube of voxels reconstructed from a single two-dimensional hologram, and it demonstrated the applicability of compressive holography to generate multidimensional images from lower-dimensional data. A detailed description of compressive holography theory and experiment results can be found in Refs. 75 and 76.
Twin-Beams DHM Method
As previously depicted in Fig. 3, the twin-beams DHM had two plane beams passing through one objective microscope. Theoretically, this process could be sketched as two cones superposed and transmitted through the particle sample P (Fig. 8, left). Beams B1 and B2 produced projections P1 and P2 of particle P on the CCD plane, respectively. In both simulation and experiment, the 3-D coordinates of B1 and B2 points and the radius of each beam circle on the CCD plane were first determined based on a simple calibration method.62 Then, the corresponding coordinates of P1 and P2 could be evaluated accordingly from the simple geometric schematic and the position of real object P was determined. A real situation of various motile cells floating into a microfluidic chamber was studied using this method and the 3-D paths of cells were estimated [Fig. 8(b)].62 The phase-contrast maps of the cells were also obtained by numerically reconstructing the recorded holograms.79,80
Applications of Three-Dimensional Profiling and Tracking by DHM
Three-Dimensional Profiling of Suspended Particles
Micro-sized particles, such as latex spheres and ferromagnetic beads, suspended in gelatin have been imaged in a 3-D volume by in-line DHM.8182.–83 A hologram of objects and another background image without the object present were both recorded. The difference of these two images eliminated the artifacts of the illumination source. Reconstruction was achieved by means of Kirchhoff-Helmholtz transform84 to generate the complex field of objects, and the combination of a stack of two-dimensional reconstructions at various planes resulted in the 3-D profile of the objects. The amplitude obtained from complex field was used to represent objects. By inspecting intensity profiles through a particle, the depth position was obtained where a sharpest intensity appeared. This allowed estimating a particle’s 3-D position coordinates within submicron. When the derivative of a second-order polynomial fitted to the intensity profiles was taken, the , , and positions of particles could be determined within 50 nm.
In Ref. 85, the distribution of dense microparticles (3.2 μm) suspended in a liquid volume with the depth of 1 mm was measured, as shown in Fig. 9. Figure 9(a) is a section of the recorded hologram and the reconstructed images at different depths are shown in Figs. 9(b) to 9(d). It was clearly observed that the in-focus particles appeared as dark circular spots. Figure 9(e) shows the combination of all the reconstructed images. The peak-searching method was then applied and each pixel was assigned the lowest intensity obtained over the entire depth. A thresholding segmentation method based on signal-to-noise ratio was applied on the reconstructed planes,86 where is the mean intensity of the particles in the volume and is the standard deviation of intensity over this volume. A list of line segments after scanning through the reconstructed images were combined to two-dimensional planar blobs using a join operator, which were then united into 3-D particle traces by repeating the peak-searching procedure at different depths. The position of a particle was then determined by using the centroid of the 3-D blob and the distribution of all the particles detected within the 3-D volume was presented in Fig. 9(f).
Three-Dimensional Study of Microfluidics
Kreuzer’s group has studied the microfluidic phenomena by in-line DHM with the capacity of large depth of field and numerical reconstruction.87 In a first experiment, a sphere of 150 μm diameter was attached to the top of a millimeter-wide tank, which was filled with water and seeded with small micro-sized latex beads as tracer. The water was sucked and pumped to flow by a blotting paper and the flow past a spherically shaped obstruction was recorded as a series of holograms by in-line DHM. The difference between consecutive hologram pairs was calculated pixel by pixel () to eliminate background structure and retain only the information of moving beads, where is the ’th hologram of recorded frames. The resulting difference holograms were summed () into a single hologram, which was then reconstructed with Kirchhoff-Helmholtz transform to obtain the focal planes of the stream lines of beads (Fig. 10). The flow information containing sequential positions of beads at successive recording times and the velocity field defined by vectors connecting two successive positions was measured and analyzed and the velocity field was proven to be in good agreement with Navier-Stokes’ solution. In this way, as indicators of flow patterns, the trajectories and velocities of latex microspheres were measured in a 3-D volume and time with subsecond and micron resolution.
The size and 3-D positions of particles in flow-through system, such as fast-moving bubbles in air-water mixture flows, were measured by Tian et al.88 Bubbles were generated by a motor driving a small propeller in a water tank and the density was not so high as to corrupt the illumination beam. Holograms were first recorded by in-line DHM system and the Fresnel approach was used to propagate the recorded object field to various planes in the vicinity of the object.55,56 At each propagation distance, the Fourier transform of the hologram was multiplied by the Fresnel approximated transfer function and an inverse Fourier transform was then applied to obtain the complex field of objects containing the intensity profile. A focused bubble appeared to have a minimum intensity and this was used as a focus criterion to locate bubbles. A two-dimensional projection image was generated by scanning the reconstructed images and recording the minimum intensity value for each pixel. The corresponding depth map where the minimum intensity occurred was also recorded for analysis. By applying thresholding on the minimum intensity projection image, a binary image showing only the edges of bubbles was created and then combined with the depth map to determine the depth of objects. The issue of overlapping edges in the projection image was resolved by using a Gaussian mixture model.89 Bubble sizes were determined from the areas enclosed by each edge. In this way, the 3-D visualization of the positions and sizes of bubbles was presented, as shown in Fig. 11.
Four-Dimensional Motility Tracking of Biological Cells
Motility is a major characteristic of certain types of cells, and it is essential to understand the motility of cells by measuring their 3-D motion. DHM offers a rapid and efficient method to monitor the 3-D motile behavior and dynamic process of various types of cells. The effectiveness of this method is quantified in terms of the maximum number and speed of the moving cells that can be tracked, as well as the direction of the motion (lateral and axial). The results from this work can be further used to implement an optical trap that can automatically track and capture cells with specified characteristics, such as speed, size, or shape.
Red blood cells
DHM has been applied in the field of hemodynamic research.90 A human red blood cells (RBC) sample was supplied to a circular microtube and the RBC motion in this microtube flow was recorded sequentially by in-line DHM. The angular spectrum method and quantification of image sharpness were applied to reconstruct and locate the focal planes of cells. The measurement accuracies in the lateral and axial directions were and , respectively. In this way, the full four-dimensional trajectories of RBCs, including space and time information, and the three-dimensional velocity field were measured, and the linear movement of cells was seen in the circular microtube. The feasibilities of in-line DHM, volumetric particle tracking, and numerical reconstruction were demonstrated with RBCs.
3-D tracking of the sedimentation of RBCs has also been implemented by DH-QPM based on the phase contrast images.91 The captured holograms were reconstructed by nondiffractive spatial-phase-shifting-based reconstruction92,93 to access to the complex field of objects, which was then numerically propagated to different planes by a convolution-based algorithm.94 The phase profile was extracted and phase unwrapping algorithm was applied for future focusing evaluation. The autofocus criterion for a sharply focused phase object showed minimum visibility.64,65 The time dependence of axial displacement and depth position where the focused phase happened were combined to illustrate the 3-D process of the sedimentation of RBC due to gravitation, as illustrated in Fig. 12.
Chilomonas are fast-moving cells, which sense and respond to their surroundings by swimming toward or away from stimuli. We have utilized off-axis DHM to track the motility of chilomonas in a 3-D volume and time.21 A time-lapse hologram movie of the movement of chilomonas (average diameter 10 μm) was first recorded at 30 fps. The field of view was with . An excerpt of 14 frames was taken and the difference between consecutive hologram pairs was calculated pixel by pixel () to eliminate background structure and retain only the object information, where is the ’th hologram of selected frames. The resulting seven difference holograms (from a total of 14) were then summed () into a single hologram [Fig. 13(a)], which contained all the information on moving cells. Reconstruction by the angular spectrum method was applied on the resultant hologram, and Fig. 13(b) was the amplitude image reconstructed at the specific plane . The 3-D trajectory of the moving chilomonas was built up by first reconstructing the two-dimensional resultant hologram from to , in steps. The autofocusing method based on peak searching was then employed, keeping the maximum intensity value at each pixel in the reconstructed image along direction to obtain the all-in-focus intensity projection [Fig. 13(c)] and combining with the depth position where in-focus intensity occurred [Fig. 13(d)] to determine the focal planes for the entire trajectory in the reconstructed volume. A threshold of intensity was applied to eliminate the unnecessary background, and the center position and the mean intensity of every point cloud representing every trajectory of chilomonas were determined. The combination of depth position and displacement of cells allowed for quantitative 3-D motility tracking. The cell was estimated to move a total path length of 93 μm at the velocity of . Figure 13(e) demonstrates the applicability of DHM for automatic four-dimensional tracking of living cells with temporal and spatial resolution at the subsecond and micro level. Figures 13(f) to 13(h) show the , , and views of the 3-D trajectory, respectively.21
In Ref. 95, a real-time hologram movie of free-swimming algae (5 to 10 μm in diameter) in seawater was recorded by in-line DHM. Subtracting the consecutive frames first and adding these difference frames then provided a final image containing the time evolution of algae trajectories free from background interference. A similar peak-searching reconstruction algorithm as described previously constructed the 3-D trajectories of algae in seawater approaching a concentrated salt solution. Free-swimming algae were seen to sense and respond to their surroundings by swimming toward or away from stimuli at an average velocity of .
Specimen of small-sized bacteria ( in diameter) inside a diatom (200 to 300 μm in diameter) was tracked in three dimensions since the diatom was almost stationary in seconds and the in-line DHM and reconstruction algorithm, including peak searching, edge detection, etc., were applicable in this weaker structure.83 This demonstrated the possibility of tracking smaller and weaker scatters with high resolution by in-line DHM.
Special cells with helical trajectories
Various self-propulsive microorganisms of different shapes with various width-to-length ratios exhibit helical swimming paths. Unlike spherical particles, these structures would cause nonuniform scattering, which makes it difficult to determine the depth map due to the large depth-of-focus of the specimen. In Ref. 96, cellular behaviors and interactions among microorganisms, such as the prey-induced changes in swimming behavior of predatory dinoflagellates, were statistically studied. The captured holograms by in-line DHM were numerically reconstructed by Fresnel transformation and the intensity field of the reconstructed images can be obtained. The same algorithm as described in Refs. 85 and 86 was utilized to determine the 3-D positions of cells. Besides this, information on each cell, such as cross-section area, dimensions, aspect ratio, total reconstructed volume, and in-focus snapshots, are stored for cell tracking. Each helical trajectory was taken as a vector determined by each criterion step by step, i.e., smoothness of the trajectory segment, 3-D velocity, acceleration, similarity of cell size, in-focus cross-section, 3-D segmented volume, shape based on correlation, aspect ratio, etc. This statistical analysis provided simultaneous tracking and characterization of cellular behavior and interactions among microorganisms in dense suspensions. Figure 14 presents the trajectories of P. piscicida before introduction of prey in a 3-D volume. In Ref. 97, another type of ellipsoidal shape Cochlodinium polykrikoides cell had a size of length and 25 μm width, and it could form chains consisting of several cells. A sample of these mixed C. polykrikoides cells, also exhibiting helical swimming paths in filtered seawater, were also statistically studied by in-line DHM and similar approaches.
Moreover, helical trajectories () of human sperm (3 to 4 μm) within a large volume () was tracked and analyzed by a dual-view and dual-color lens-free DHM setup, which consisted of two partially coherent LEDs of different wavelengths placed at 45 deg with respect to each other (Fig. 15).98 The recorded in-line holograms only reconstructed at correct combination of depth, angle, and wavelength could produce clear images, and the sperm head images projected at two different wavelengths and viewing angles could be isolated even if they were recorded in the same frame. Combining with iterative phase recovery method,99 the peak-searching algorithm, and related numerical smoothing method, the 3-D trajectories of sperms and the statistics information of the swim path, pattern, speed, etc., were revealed (Figs. 15 and 16).
Crawling cells in matrix gels
Ferraro’s group has presented a 3-D tracking of in vitro osteosarcoma cells in a gelified collagen matrix characterized by a high morphological change.100 DH-QPM was utilized to capture time-lapse sequences of cell migration at 12 fph for 15 h to record any movement and morphological change of cells. Numerical reconstruction allowed for evaluating both intensity and phase profiles in different planes. The cells were first detected from phase profiles by applying a simple thresholding filter, and this type of pure phase objects showed minimum visibility when they were in focus; therefore, the estimation of focal planes was performed by numerical scanning on the stack of reconstructed amplitude images instead for the sharpest details. Then, the axial position of the center of mass of each cell was determined from phase profiles at the focal plane.101 Taking into account the morphological changes of cells, the concept of minimum boundary filters100 was introduced to determine the positions of cells, and it was compared with other three methods, i.e., centroid, weighted centroid, and maximum phase value, as shown in Fig. 17.
In Ref. 102, to reduce the coherent noise, a partially spatial coherent illumination was utilized as the illumination source of off-axis DHM setup and an accurate equalization between the object and reference beams must be reached. A semiautomated software package performed a four-dimensional tracking of unstained HT-1080 fibrosarcoma cells migration in matrix gels on the basis of phase profiles, and the 3-D locations of a central point in the cell as a function of time were recorded simultaneously. Quantitative measures on the displacement and velocity of cells were also obtained. By numerical autofocusing and evaluation of quantitative phase images, DH-QPM was demonstrated to be a label-free and noninvasive method for tracking living cells in a 3-D environment.
Characterization of Microfibers
Kempkes et al. have applied in-line DHM to record a hologram of a needle-shaped tilted opaque carbon fiber103 and showed the reconstruction at different planes using Fresnel-Kirchhoff diffraction integral.94 The intensity profiles were then obtained and converted to binary images by setting a threshold, which retained only the in-focus pixels. The combination of the stack of binary images produced a 3-D profile containing point clouds resulting from the focused pixels of a fiber at different reconstructed planes. A straight line was fit onto the point clouds, and the orientation and length of the fiber were estimated. A more complex sample of fibers randomly suspended in water was studied in the same way. Seven fiber segments were identified and the corresponding orientation and length were measured (Fig. 18). In-line DHM is shown to be an effective technique to characterize nonspherical particles in 3-D volume.
We have recently studied the nonwoven curved and randomly oriented microfibers in suspension.21 Fibers were spun onto glass coverslips from co-poly (, ) (PLEY) dissolved in water and crosslinked as described previously.58 The diameter of fibers was 0.2 to 5.0 μm. Figure 19(a) presents a hologram of microfiber. The angular spectrum method was applied to obtain the reconstructed amplitude image in Fig. 19(b). The microfiber appeared curved and elongated, consistent with our previous work.104 The field of view was with . The hologram was then reconstructed from to , in steps. Adding these 201 reconstructed images of amplitude yielded the axial projection shown in Fig. 19(c). The final image had the same number of pixels as each reconstructed image. The projected image was converted to binary by threshold and segmentation, as shown in Fig. 19(d). The binary image was then axially backprojected along the path of amplitude over the set of reconstructed images. We applied the same autofocusing algorithm on the resultant images to record the maximum intensity value at each pixel along the reconstruction direction and the axial and depth position of where the maximum intensity occurred at the same time. The obtained all-in-focus intensity profile [Fig. 19(e)] and the corresponding depth map [Fig. 19(f)] were then combined to generate the 3-D visualization of microfiber and a threshold on intensity was chosen to highlight the area of regional interest. However, there still existed noisy point clouds from speckle noise in the 3-D profile [Fig. 19(g)]. To address this, an average and polynomial curve fitting algorithm accounting for errors was adopted. The fitted line of the point clouds (standard deviations in the and directions were 0.21 and 0.06 μm) in the 3-D coordinate was present in Fig. 19(h). Figures 19(i) to 19(k) show the , , and views of the 3-D profile, respectively. coordinate provides the depth information of real image of microfiber in the reconstruction volume. The length of the microfiber in 3-D volume was determined from , , and coordinates of real image of microfiber, which was 126 μm.21
When cells swim in the local environment, they exhibit movement toward or away from the stimuli, as well as interaction with the stimuli. Knowledge of this interaction is essential for understanding the impact of cell’s behavior on the process of locating food, avoiding predators, finding mates, etc.
Three-dimensional flow field by swimming copepod
In-line DHM was utilized to measure the 3-D trajectory of a free-swimming copepod and the complex flow around it.105 A sample of Diaptomus minutus in well water was prepared and seeded with 20 μm polystyrene spheres. The in-line DHM apparatus with He-Ne laser source consisted of two inclined mirrors on the walls of the sample volume to provide two perpendicular views of the object in the same image. The series of holograms that captured the copepod with two views were reconstructed using Fresnel-Huygens principle106 with a paraxial approximation to obtain the intensity distribution. Based on the location of the traces resulting from the superposition of three successive exposures, the corresponding location of mirrored views was estimated. With each of the two views, the particle displacement was measured using cross-correlation of the intensity distribution of the three exposures,107,108 and the lateral and vertical displacements were then combined into a 3-D velocity distribution of particles, as shown in the left panel of Fig. 20. In the reconstructed images sequence movie, the copepod was seen to sink, swim upward, and then sink again. As it sinks, a recirculating flow pattern was generated due to the movement of appendages, as illustrated in the right panel of Fig. 20. Stokeslets model109,110 was employed to estimate the excess weight () and excess density () of the copepod, and also the propulsive force generated by appendages ().
Three-dimensional displacement of microfiber by swimming paramecium
A hologram movie of paramecium moving through a microfibers mesh was recorded by off-axis DHM at 17 fps.21 A cell was seen to swim through fibers and cause obvious displacement of fibers. Two hologram frames showing cell approaching to fibers [Fig. 21(a)] and swimming away after pulling fibers [Fig. 21(b)] were extracted. The field of view was with . The difference hologram [Fig. 21(c)] clearly showed the displacement of the fiber due to the movement of cell and the trajectory of cell. The reconstructed amplitude images from angular spectrum method at and are shown in Figs. 21(d) and 21(e), respectively. Different parts of the fiber appeared to be in focus at different reconstructed planes. Autofocusing algorithm at the reconstruction volume from to was applied and the all-in-focus intensity and depth position information were obtained in Figs. 21(f) and 21(g).
To best visualize the displacement of the fiber in 3-D volume, we analyzed the two tracks of the displacement of the fiber independently. For brevity, most of the following descriptions referred to the first track. Numerical segmentation and thresholds were applied to Fig. 21(f), and the all-in-focus intensity image containing only one track of fiber was generated in Fig. 21(h). Combined with the obtained depth position profile [Fig. 21(g)] and fitting algorithm accounting for errors, the fitted line of the point clouds (standard deviations in and directions were 1.27 and 0.27 μm) in 3-D coordinate is present in Fig. 21(i). Similar results for the second track of fiber were shown in Figs. 21(j) and 21(k), and the standard deviation of its corresponding point clouds in the and directions were estimated to be 0.80 and 0.26 μm. The two tracks of the fiber were then plotted in one 3-D coordinate [Fig. 21(l)], and Figs. 21(m) to 21(o) show the , , and views of the 3-D plot, respectively. The length and the displacement of the fiber were estimated to be 210.1 and 11.1 μm. A movie of 3-D displacement of the fiber within 0.5 s period was also presented. The 3-D displacement of an individual microfiber due to interaction with paramecium cell has been measured as a function of time at subsecond and micrometer level.21
To the best of our knowledge, this is the first quantitative profiling and tracking by DHM of the curvature and displacement of individual microfiber by swimming cells in three dimensions. Displacement could vary with cell type, physiological state of the cells, microfiber preparation, etc. The prospects seem excellent for DHM for particle flow analysis and 3-D imaging of randomly oriented microfibers, in cases where quantitative measurement of characteristics (size, length, orientation, speed, displacement, etc.) is of great interest.
We present a review of a subset of DHM for 3-D profiling and tracking, focusing on techniques, reconstruction criteria, and applications in various fields. For brevity, a summary of the research progress is provided in Table 1. The table is divided into studied objects and their status, and also includes the DHM techniques used for imaging and the corresponding reconstruction methods and autofocusing algorithms. DHM is demonstrated to have the capacity of full-field, label-free, noncontact, noninvasive, real-time refocusing of an object without mechanical focus adjustment and proven to be a potent tool to perform 3-D imaging and tracking. The combination of DHM with robust autofocusing algorithms enables the remarkable study of 3-D profiling and tracking of micro-objects or nano-objects. It has merits of wide applications covering particles, living cells, and fibers with different shapes (spherical, needle shaped, and randomly oriented), size (few to hundred micrometer), and conditions (static, suspended, and flow-through). Due to the uncertainties connected with the imaging target in 3-D profiling and tracking, it is important to tailor and find strategies that can be adapted to the specific problem.
Summary of research progress in the field of digital holographic microscopy (DHM) for three-dimensional profiling and tracking.
|Object||Status of object||DHM configuration||Method of reconstruction & autofocusing algorithms|
|Microspheres/beads Suspendesus||Suspended, sparse||Off-axis||Angular spectrum sharpness and peak searching21|
|Suspended, sparse||In-line||Kirchhoff-Helmholtz transform sharpness and peak searching81–83|
|Suspended, sparse||In-line||Three-dimensional deconvolution71|
|Suspended, sparse||In-line||Rayleigh-Sommerfeld backpropagation72|
|Suspended, dense||In-line||Fresnel sharpness and peak searching85|
|Flow-through||In-line||Kirchhoff-Helmholtz transform sharpness and peak searching87|
|Seed parachutes||Separated position||In-line||Compressive holographic method75|
|Bubbles||Flow-through||In-line||Fresnel sharpness and peak searching88|
|In vitro cells||Floating||Twin-beams||Twin-beams DHM method62|
|Chilomonas (10 μm)||Free-swimming||Off-axis||Angular spectrum sharpness and peak searching21|
|Algae (5 μm)||Free-swimming||In-line||Kirchhoff-Helmholtz transform sharpness and peak searching95|
|Bacteria (<1.5 μm)||Free-swimming||In-line||Kirchhoff-Helmholtz transform sharpness and peak searching83|
|Cochlodinium polykrikoides cell (25 to 40 μm)||Free-swimming||In-line||Angular spectrum sharpness and peak searching97|
|Red blood cell||Flow-through||In-line||Angular spectrum sharpness and peak searching90|
|Sedimentation||DH-QPM||Fresnel–Kirchhoff integral sharpness and peak searching91|
|Predatory dinoflagellates||Helical motion||In-line||Kirchhoff-Helmholtz transform sharpness and peak searching96|
|Human sperms||Helical motion||Two-set in-line with LEDs||Angular spectrum, iterative phase recovery sharpness and peak searching98|
|Osteosarcoma cells||Crawling on matrix gels||DH-QPM||Fresnel–Kirchhoff integral sharpness and peak searching100|
|HT-1080 fibrosarcoma cells||Crawling on matrix gels||DH-QPM with LED illumination||Fresnel–Kirchhoff integral sharpness and peak searching102|
|Microfibers||Straight, suspended||In-line||Fresnel–Kirchhoff integral sharpness and peak searching103|
|Curved, suspended||Off-axis||Angular spectrum sharpness and peak searching21|
|Copepod, microsphere||Interaction of cell and flow field||In-line||Fresnel-Huygens sharpness and peak searching105|
|Microfibers, paramecium||Interaction of fiber and cell||Off-axis||Angular spectrum sharpness and peak searching21|
Note: DH-QPM, quantitative phase microscopy by digital holography.
The past years have seen DHM develop to such a state that it has become a potent tool to be considered in the various applications. Indeed, DHM revitalizes the general area of holography and is seen to be a breakthrough in microscopic imaging and application. Researchers not only in optical field, but also in microbiology, medicine, marine, metrology, etc., are realizing the new capabilities of DHM. Although it is necessary to develop and improve the performance, such as noise suppression, resolution enhancement, to name but one, we believe that with the development of optical imaging and computing techniques, DHM will be more diversified to solve different specific problems with appropriate strategies, and it deserves special attention to the future contributions in the field of energy, chemical activity, clinical behavior, remote sensing, environmental monitoring, etc.
Xiao Yu received a BS degree in physics from Nankai University, Tianjin, China, in 2008 and MS degree in applied physics from University of South Florida (USF), Tampa, Florida, in 2010. She is currently working toward her PhD degree at USF. Her main research interests are digital holographic microscopy and its applications in biological field, biomedical imaging, digital image processing, etc.
Jisoo Hong received his BS and MS degrees in electrical engineering from Seoul National University, Korea, in 2002 and 2004, respectively. From 2004 to 2008, he was a senior research engineer with LG Electronics, Korea. In August 2012, he received his PhD degree from his alma mater. Presently, he is a postdoctoral researcher in the Department of Physics at USF, Tampa, Florida. He is interested in incoherent digital holography and its application to biomedical imaging and three-dimensional photography.
Changgeng Liu obtained his BS in applied physics in 2007 and his MS in optics in 2010 from Beijing University of Technology. He is currently seeking a PhD degree in applied physics in USF. His research interests include digital holography, adaptive optics, biomedical imaging, computational optics, phase retrieval, digital imaging processing, and optical systems design.
Myung K. Kim is a professor of physics at the USF, and directs the Digital Holography and Microscopy Laboratory. He received his PhD degree from the University of California, Berkeley, in 1986. His current research interests are in the development of novel techniques and applications in digital holography, microscopy, interference imaging, optical tomography, biomedical imaging, and manipulation of particles and microbes by optical force. He has more than 200 publications and 13 patents granted or pending.