Typical LED array microscopes require multiple image acquisitions for phase retrieval. Here, we propose a polarized LED array microscope for single-shot quantitative phase imaging with aberration correction. We implemented polarization-encoded illumination multiplexing by placing a custom-made polarization filter on top of the LED array. A single image was captured using a polarized sensor under polarized LED illumination. We reconstructed the quantitative phase by incorporating the polarization multiplexing model with a phase retrieval algorithm. We showed that the proposed technique can reconstruct aberration-corrected phase images with a single-shot intensity image.
KEYWORDS: Deep learning, Tissues, In vivo imaging, Endoscopy, Education and training, Diseases and disorders, Biopsy, Biological samples, Tissue optics, Neural networks
Conventional imaging techniques target this problem by using specific antibody markers. Although such markers allow decent specificity, they are often limited in the field of application, especially for in vivo use, which limits the potential for clinical translations. In contrast to that, label-free optical technologies, like multiphoton microscopy (MPM), can generate highly resolved 3D images from unstained samples, by exploiting natural optical contrast. Label-free MPM can show epithelial damage and immune infiltration in unstained colon samples. Here, we imaged a mixture of T cells and neutrophils with label-free MPM. In order to obtain ground-truth images, we simultaneously recorded images of a Cd4+ specific fluorescent marker for T cells. A deep neural network was then trained for the segmentation of T cells and neutrophils based on such label-free MPM images. Upon training, this model can then be used to detect both cell types without relying on specific fluorescent markers, that were used to obtain ground truth. In the future, the augmentation of label-free MPM by such computational specificity could have great potential for in vivo endomicroscopy.
We present a novel polarization-sensitive Fourier Ptychographic Microscopy (FPM) method that leverages multiplexing techniques in the Fourier plane, eliminating the need for costly polarization cameras or mechanical polarizer rotations. By simply introducing semicircular 0° and 90° linear polarizers in the Fourier plane of a conventional FPM setup, we can effectively split a single pupil into two half-circle pupils, enabling the simultaneous multiplexing of two channels' signals within a single measurement. By imposing two pupil functions on FP phase retrieval, we reconstructed the amplitude and phase information of the two orthogonal polarization channels, ultimately obtaining the Jones matrix of the anisotropic specimen. To validate our proposed method, we demonstrate its application by accurately reconstructing the orientation of the slow axis and phase retardation of MSU crystals known as the birefringence object.
We propose a high-throughput phase-guided digital histological staining based on Fourier ptychographic microscopy using a generalizable deep neural network. Since the phase information includes the refractive index distribution of the specimen, we can digitally stain the unstained tissue slides from the quantitative phase images, which present the same color features that can be observed under a conventional microscope with the staining process. Here, we utilize Fourier Ptychographic Microscope that enables wide field and high-resolution quantitative phase imaging using multiple measurements by varying illumination angle. Additionally, we design a neural network that has remarkable generalization regarding sample dependence with the learned forward model. Along with this network architecture, we realize the efficient and effective digital staining process that does not require the labeled dataset from unstained tissue slides. We will report on the digital stained result from raw FPM images, the performance comparison, and discuss the future direction of our approach.
KEYWORDS: Organisms, Microscopes, Tomography, Algorithm development, 3D modeling, Detection and tracking algorithms, 3D tracking, 3D image processing, Reconstruction algorithms, Muscles
It is challenging to study behavior of and track freely-moving model organisms using conventional 3D microscopy techniques. To overcome motion artifacts and prevent the organism from leaving the field of view (FOV), existing techniques require paralyzing or otherwise immobilizing the organism. Here, we demonstrate hemispheric Fourier light field tomography, featuring a parabolic objective that enables synchronized multi-view fluorescence imaging over ~2pi steradians at up to 120 fps and across multi-millimeter 3D FOVs. Our method is not only able to track the 6D pose of freely-moving zebrafish and fruit fly larvae, but also other properties such as heartbeat, fin motion, jaw motion, and muscle contractions. We also demonstrate simultaneous multi-organism imaging.
We present rolling shutter speckle imaging (RSSI), a single-shot temporal speckle imaging technique that can quantitatively measure the fast dynamics of scattering media without high speed cameras. Utilizing a rolling shutter image sensor and vertically elongated speckles, RSSI can quantitatively map the speckle dynamics from a single image capture. We discuss the speckle spatiotemporal intensity correlation model for RSSI, which is validated through simulations and phantom experiments. We show in vivo quantitative blood flow imaging of the mouse brain from a snapshot measurement. In addition to imaging, we also present rolling shutter speckle plethysmography for cardiovascular monitoring.
We report tensorial tomographic Fourier ptychography (T2oFu), a nonscanning label-free tomographic microscopy method for simultaneous imaging of quantitative phase and anisotropic specimen information in 3D. Built upon Fourier ptychography, a quantitative phase imaging technique, T2oFu additionally highlights the vectorial nature of light. The imaging setup consists of a standard microscope equipped with an LED matrix, a polarization generator, and a polarization-sensitive camera. Permittivity tensors of anisotropic samples are computationally recovered from polarized intensity measurements across three dimensions. We demonstrate T2oFu’s efficiency through volumetric reconstructions of refractive index, birefringence, and orientation for various validation samples, as well as tissue samples from muscle fibers and diseased heart tissue. Our reconstructions of healthy muscle fibers reveal their 3D fine-filament structures with consistent orientations. Additionally, we demonstrate reconstructions of a heart tissue sample that carries important polarization information for detecting cardiac amyloidosis.
We report on the design and construction of a goggle-type eye tracker using a low-cost and high-speed lensless camera for monitoring eye movements in neurodegenerative diseases. A Rolling Shutter image sensor combined with lensless computational imaging allows for the reconstruction of a time sequence of images from a single snapshot, effectively improving the framerate of the camera. We constructed and demonstrated the prototype device using a commercial-grade CMOS image sensor and achieved the improvement of framerate from 15 to 480Hz, with the tracking results for 28 clinical measured data. Our device can potentially measure microsaccadic eye movements in a wearable camera format, allowing routine monitoring of abnormal eye movements for the early diagnosis and tracking of Alzheimer’s and Parkinson’s disease.
“Anyone who uses a microscope has likely noticed the limitation of the trade-off between the field of view and the resolution”. To acquire highly resolved images at large fields of view, existing techniques typically record sequential images at different positions and then digitally stitch composite images. There are alternatives to this mechanical scanning procedure, such as structured illumination microscopy or Fourier ptychography that record sequential images at varying illuminations prevent mechanical scanning for high-resolution image composites. However, all of these approaches require sequential images and thus compromise speed, temporal resolution and experimental throughput. Here we present the Multi-Camera Array Microscope (MCAM), which is a microscope system that utilizes an array of many synchronized cameras, each with an individual imaging lens, for simultaneous image capture. The MCAM enables enhanced imaging capabilities and novel applications in various scientific and medical fields, by combining the images acquired from each individual camera-lens pair.
Fourier ptychography (FP) utilizes angle-varied illumination to achieve resolution improvement and quantitative phase imaging. In this talk, we present a compact microscope using an OLED screen as a programmable illumination for FP reconstruction. We discuss multiplexed reconstruction strategy using multi-pixel illuminations, and a stand-alone smartphone implementation of portable FPM.
We present a high-throughput computational imaging system capable of performing dense, volumetric fluorescence imaging of freely moving organisms at up to 120 volumes per second. Our method, termed 2pi Fourier light field tomography (2pi-FLIFT), consists of a planar array of 54 cameras and a parabolic mirror serving as the primary objective that allows for synchronized multi-view video capture over ~2pi steradians. 2pi-FLIFT features a novel 3D reconstruction algorithm that recovers both the 3D fluorescence distribution and attenuation map of dynamic samples. We demonstrate 2pi-FLIFT on important, freely moving model organisms, such as zebrafish and fruit fly larvae.
Lensless photography is a recently-developed computational imaging technique that uses light-modulating phase masks instead of lenses to build ultra-compact and low-cost cameras. Here, we propose a method to design and fabricate custom phase-masks to create deterministic point spread functions for lensless imaging. Phase-masks are designed using our wave-optics-based algorithm and are fabricated in single-shot via grayscale maskless lithography technique. Using this method, we can design and fabricate phase-masks with various surface profiles and have validated that the fabricated masks match the designed profiles. Our technique allows for a fast and efficient process that can be applied to the fab-level fabrication of lensless cameras for commercialization. We also show that lensless cameras using our custom phase-masks can effectively obtain images in a compact form factor.
Fourier Ptychographic Microscopy (FPM) is a computational imaging technique which reconstructs super-resolved amplitude and phase images by combining variably illuminated low-resolution images through an iterative phase retrieval algorithm. However, the phase-retrieval-based reconstruction requires sufficient overlap between spatial frequency bands of the measurements, which creates a trade-off between the number of measurements and the reconstruction quality. We propose a deep-learning-based FPM reconstruction that recovers both amplitude and phase images in high resolution with far fewer measurements than conventional FPM, with model-based constraint. Our model works with almost no overlap between low-resolution measurements in the Fourier domain, only taking into account the total Fourier extent of the measurements.
We propose a simple smartphone attachment module to realize a portable wide-field high-resolution microscope based on Fourier Ptychographic Microscopy (FPM). Using the smartphone's screen as the illumination and the front camera module for image acquisition, we can construct a stand-alone portable FPM, a microscopy technique that can achieve high resolution by computationally combining a number of variably illuminated low-resolution bright-field and dark-field images through an iterative phase retrieval algorithm. With the custom-built android application that performs in situ calculation for acquisition, reconstruction, and display of the images, we can achieve a true stand-alone portable imaging device for field applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.