This PDF file contains the front matter associated with SPIE Proceedings Volume 11250, including the title page, copyright information, table of contents, and author and conference committee lists.

Selective-plane illumination microscopy (SPIM) are particularly advantageous in long-term live volumetric imaging, although its common geometry is incompatible with common biological sample holders, including multi-well plates. To solve this problem, we designed an epi-illumination SPIM (eSPIM) system with high spatial resolution and light collection efficiency with single-molecule sensitivity. It has an identical sample interface as an inverted fluorescence microscope with a single primary objective and no additional reflection elements. We have demonstrated multicolor, fast, volumetric imaging of live cells as well as single-molecule super-resolution microscopy on eSPIM. Moreover, it has enabled long term imaging of cells in parallel in multiwell plates and simultaneous recording of cellular responses to a panel of perturbations.

Light-field microscopy has emerged as a particularly elegant and powerful technique for fast and instantaneous three-dimensional imaging in biology, as it captures a 3D image with a single camera snapshot. In this talk I will present recent work in which we tackled the main challenges of establishing this technique for applications in the life sciences. In particular I will show how innovative approaches have allowed us to increase spatial resolution and contrast while reducing the presence of artefacts, all of which were imperative for imaging fast biological processes such as neuronal and cardiovascular dynamics.

We discuss two compact, cost-effective, and field-portable ptychographic lensless imaging platforms for quantitative microscopy. In the first implementation, we use a low-cost galvo scanner to rapidly scan an unknown laser speckle pattern on the object. To address the positioning repeatability and accuracy issues, we directly recover the positional shifts of the speckle pattern based on the phase correlation of the captured images. To bypass the resolution limit set by the imager pixel size, we employ a sub-sampled ptychographic phase retrieval process to recover the complex object. In the second implementation, we place a thin diffuser in between the object and the image sensor for light wave modulation. By blindly scanning the unknown diffuser to different x-y positions, we acquire a sequence of modulated intensity images for quantitative object recovery. Different from previous ptychographic implementations, we employ a unit magnification configuration with a Fresnel number of ~50,000, which is orders of magnitude higher than previous ptychographic setups. The unit magnification configuration allows us to have the entire sensor area, 6.4 mm by 4.6 mm, as the imaging field of view. The ultra-high Fresnel number enables us to directly recover the positional shift of the diffuser in the phase retrieval process. In this second implementation, we use a low-cost, DIY scanning stage to perform blind diffuser modulation. We further employ an up-sampling phase retrieval scheme to bypass the resolution limit set by the imager pixel size and demonstrate a half-pitch resolution of 0.78 µm. For both implementations, we validate the imaging performance via various biological samples. The reported platforms provide cost-effective and turnkey solutions for large field-of-view, high-resolution, and quantitative on-chip microscopy. They are adaptable for a wide range of point-of-care-, global-health-, and telemedicine-related applications.

Cell type classification and isolation according to imaging and spatial characteristics, beyond traditional fluorescently labeled biomarkers, enable the development of new biological insight and establishment of connections between phenotypical, morphological, and genomic cell information in normal and diseased states. Here we demonstrate a 2D image-guided cell sorter and a 3D imaging flow cytometer using fast scanning laser excitation sources. Both systems feature a cameraless design, which reconstructs cell images from the temporal readout of photomultiplier tubes.

Identification of different cell types is an indispensable part in biomedical research and clinical application. During the last decades, much attention was put onto molecular characterization and many cell types can now be identified and sorted based on established markers. The required staining process is a lengthy and costly treatment, which can cause alterations of cellular properties, contaminate the sample and therefore limit its subsequent use. A promising alternative to molecular markers is the label-free identification of cells using mechanical or morphological features. We introduce a microfluidic device for active label-free sorting of cells based on their bright field image supported by innovative real-time image processing and deep neural networks (DNNs). A microfluidic chip features a standing surface acoustic wave generator for actively pushing up to 100 cells/sec to a determined outlet for collection. This novel method is successfully applied for enrichment of lymphocytes, granulo-monocytes and red blood cells from human blood. Furthermore, we combined the setup with lasers and a fluorescence detection unit, allowing to assign a fluorescence signal to each captured bright-field image. Leveraging this tool and common molecular staining, we created a labelled dataset containing thousands of images of different blood cells. We used this dataset to train a DNN with optimized latency below 1 ms and used it to sort unstained neutrophils from human blood, resulting in a target concentration of 90%. The innovative approach to use deep learning for image-based sorting opens up a wide field of potential applications, for example label-free enrichment of stem-cells for transplantation.

Multiplexed asymmetric-detection time-stretch optical microscopy (multi-ATOM) has recently been developed to enable high-throughput quantitative phase imaging flow cytometry, from which single-cell biophysical properties can be measured at large scale. However, it lacks the ability to link such biophysical knowledge to biomolecular signatures at the single-cell precision for validation and correlative multi-scale single-cell analysis. We report a high-throughput multimodal system that integrates multi-ATOM with multiplexed 1-D fluorescence imaging/detection, termed FluorATOM; and applied it to perform synchronized biophysical and biomolecular phenotyping of rare breast circulating tumor cells detected in peripheral blood in a mouse xenograft at a throughput of >10,000 cell/sec.

Membrane nanodomains have commonly been implicated in biological processes. However, what these nanodomains are and how they participate in the processes of interest are still unclear, primarily due to challenges in probing these nanoscopic and dynamic structures in cells. Using high-throughput single particle tracking via spt-PALM and detailed trajectory analysis, here we demonstrate that membrane nanodomains associated with the small GTPase KRas could be detected and analyzed in live cells. By stochastically activating and tracking single PAmCherry1-KRas molecules on the membrane, spt-PALM yields 5,000 to 100,000 single-molecule diffusion trajectories of KRas. Analysis of these trajectories with variational Bayes SPT (vbSPT) revealed that KRas exhibits an immobile state in domains ~70 nm in size, each embedded in a larger domain (~200 nm) that confers intermediate mobility, while the rest of the membrane supports fast diffusion. By analyzing the transition kinetics among the three states, we found that KRas is continuously removed from the membrane via the immobile state and replenished to the fast state, likely coupled to internalization and recycling. Our results demonstrate the utility of high-throughput SPT in uncovering the impact of nanoscopic landscape of the membrane on the spatiotemporal dynamics and potentially multimer formation and signaling of membrane-bound biomolecules.

Recently, Beier et al. a new imaging system called a streak camera microscope (SCM) that is able to resolve sub nanosecond membrane potential changes in cells loaded with FluoVolt dye. This technique opens the window into the response of cell membrane potential to rapidly applied electric fields. Having demonstrated this response on single bipolar and unipolar electric pulses, we have begun to investigate the rapid charging and discharging of the plasma membrane during bursts of AC frequencies. We believe that understanding the dynamics of plasma membrane charging during AC pulses will better inform those using such pulses to modify cell behavior.

In tumor resection surgery, the entire tumor must be removed to prevent local recurrence of cancer. To achieve effective and successful tumor resection surgery, an intraoperative examination is performed for quick decision-making during the surgical process. Examination of frozen sections is a common method, but it has limitations that it requires time-consuming tissue processing procedures which leads to interpretation errors. Photoacoustic microscopy (PAM) with ultraviolet (UV) laser is a promising intraoperative surgical margin assessment method that provides depth-resolved and label-free imaging of cell nuclei without sectioning and staining. Despite these advantages, conventional PAM still has limited imaging speed that does not allow real-time imaging, because it achieves the volumetric images by raster scanning using 2-axis step motors. To overcome the limitation, we developed a high-speed reflection-mode OR-PAM based on a UV scanner. Using the scanner module, it took 180 seconds to acquire one volumetric data over 1 × 1 mm². In an in-vitro test, the measured lateral and axial resolution were 1.2 μm and 65.1 μm, respectively. We performed ex-vivo experiments on paraffin sections of tissues after deparaffinization that had been excised from a kidney, liver, colon-cancer and a liver-cancer patient. We could find structures in tumorous tissues distinguishable from normal tissues in 4 × 8 mm² which is clinically meaningful FOV. We could also identify single nucleus in UV-PAM images, and match it with the corresponding nucleus in microscopic images.

We develop a high speed compressive Raman imaging technology using a programmable binary spectral filter and a single channel detector to perform fast Raman detection and concentration estimation of know species over millimeters field of view. The technology is x100 times faster than commercial CCD based systems and x10 times faster than the EMCCD based systems. We report fast imaging of breast micro-calcifications, SERS particles, pharmaceutical tablets and micro-plastics. We also present a novel fast line scan compressive Raman imaging technique using spatial frequency-modulated illumination (SPIFI) that enables to encode space into the frequency domain to acquire single shot line images. We demonstrate the imaging and classification of three different chemical species at line scan rates of 40 Hz.

Innovations in optical spectroscopy and microscopy have revolutionized our understanding in biological systems. In this talk, I will discuss our recent development by coupling stimulated Raman scattering (SRS) microscopy with chemical probes that could allow high-sensitivity bio-analysis with fast speed at the sub-cellular level. Both physical and chemical principles underlying the optical microscopy will be presented, as well as our efforts in biomedical applications including cancer- and neuronal- metabolism.

We present a high-throughput multimodal Raman-fluorescence flow cytometer that produces simultaneous Raman and fluorescence excitation spectra at a demonstrated throughput of 100 events per second. Our method combines Fourier-transform coherent anti-Stokes Raman scattering with Fourier-transform two-photon excitation fluorescence, which produces broadband Raman spectra that span the biological fingerprint region and single-detector multicolor fluorescence via spectral analysis. As a proof-of-principle demonstration, we perform multimodal label-free analysis of stress response in the microalga Haematococcus pluvialis by Raman measurement of its secondary metabolite astaxanthin and two-color fluorescence measurement of chlorophyll and fluorescent chlorophyll catabolites under different stress conditions.

Single molecule localization microscopy (SMLM) is one of the most popular super-resolution imaging methods. In this talk, we'll highlight recent computational developments of our lab to push the limits of SMLM. First, we will present ANNA-PALM, a computational technique based on deep learning that can reconstruct high resolution views from strongly under-sampled SMLM data and widefield images, enabling considerable speed-ups without any compromise on spatial resolution. Second, we will present ZOLA-3D a combined optical and computational method that enables versatile 3D super-resolution imaging over up to ~5 um depth. Third, we will briefly highlight shareloc.xyz, an online platform to facilitate the sharing and reanalysis of SMLM data.

Fluorescence microscopy lets biologist see and understand the intricate machinery at the heart of living systems and has led to numerous discoveries. Any technological progress towards improving image quality would extend the range of possible observations and would consequently open up the path to new findings. I will show how modern machine learning and smart robotic microscopes can push the boundaries of observability. Moreover, I will demonstrate how smart microscopy techniques can achieve the full optical resolution of light-sheet microscopes — instruments capable of capturing the entire developmental arch of an embryo from a single cell to a fully formed motile organism.

Platelets participate in both physiological hemostasis and pathological thrombosis by forming aggregates activated by various agonists. However, it has been considered impossible to identify the stimuli and classify the aggregates. Here we present an intelligent method for classifying platelet aggregates by agonist type based on the combination of high-throughput imaging flow cytometry and a convolutional neural network. It morphologically identifies the contributions of different agonists to platelet aggregation with high accuracy. The method is a powerful tool for studying the underlying mechanism of platelet aggregation and is expected to develop a new class of clinical diagnostics and therapeutics.

Label-free identification of immune cells presents an outstanding challenge in the current era of advanced technologies. For this, optical techniques of Raman spectroscopy and digital holographic microscopy (DHM) have been devised to successfully identify the immune cells. For accurate classification, these techniques require a post processing step of linear methods of machine learning. In this study, we show a comparison of principal component analysis and artificial neural networks for the classification of neutrophils and eosinophils based on Raman spectroscopic data and DHM based microscopic data. We show that DHM when combined with convolutional neural networks proves to be a robust, stand-alone and high throughput hemogram with a classification accuracy of 91.3% at a throughput rate of more than 100 cells per second.

We present an extreme-throughput (>1 million cells per second) imaging flow cytometer with deep learning to achieve a highly simple, rapid, and cost-effective liquid biopsy for ex-vivo drug-susceptibility testing of leukemia. The drug resistance of leukemia cells was detected in whole blood with only 24-hour drug treatment without hemolysis or dilution, making the sample preparation extremely simple, rapid and cost-effective. Our method also accurately evaluated the drug susceptibility of white blood cells from untreated patients with acute lymphoblastic leukemia, holding great promise for affordable precision medicine.

Flow cytometers are invaluable tools that can quantitatively analyze and separate cells with respect to a cell’s biophysical and biochemical properties. Conventional cytometers collect these physical and chemical properties in the forms of inelastic light scatter and fluorescence. Specialized cytometers came to fruition after several advancements; smaller, more efficient photodetectors, tunable laser diodes, and the advent of microfluidics. Our work focuses on the latter topic. Microfluidic-based flow cytometry is robust in single cell and single molecule detection. Recent studies have leveraged significant quantitative analysis from multiplexing in phenotyping experiments, rare events in highcontent screening assays and sorting. Multiplexing requires multiple color channels to capture and resolve the presented spectral data. Color compensation is needed to resolve emission spectra overlap and becomes difficult when 10+ colors are used. Rare event detection requires large volumes of sample to the effect of 109 cells and greater. The task becomes time and resource consuming because conventional flow is limited by linear flow velocities (50,000 events/second) and requires extensive amounts of sheath fluid. Lastly, collecting these events by conventional flow requires careful separation by means of fluorescence activated cell sorting (FACS). Most cell sorters are capable of high yields but use piezoelectric transducers that are not as biocompatible as once thought. Herein we present a time-resolved acoustofluidic flow cytometer that can theoretically surpass the linear velocity constraint, use acoustic focus to elevate biocompatibility and reduce resource consumption and eliminate the need for multiple color channels.

We have developed compressed ultrafast holography (CUH) to reconstruct the optical field of dynamic scenes in a single camera snapshot. In CUH, each row of the phase mask on the reference beam has a distinct carrier frequency. When the object interferes with the phase mask, each row will be shifted to a different location in the Fourier domain. Then, the interfered object and reference beam is swept across a camera during a single exposure to encode the temporal information. Since these shifts in Fourier domain will be overlapping, compressive sensing is applied to reconstruct the dynamic scenes.

High-resolution image-based observations of 3D motion from multiple objects are useful for individual identification and monitoring of crowds of objects/creatures. However, using only a wide-angle camera does not allow suitable observations given the tradeoff between angle of view and resolution. In contrast, a high-speed gaze controller and a high-speed variable focus lens can provide image recognition with high-resolution observations. On the other hand, simultaneous control can be inefficient for single-target tracking and multiple-target switching against automatic gaze/focus scanning. Therefore, we propose a hybrid high-speed gaze/focus control tracking system and cooperative operation with a wide-angle camera. Triangulation from the tracking system and wideangle camera based on a fundamental matrix and a lookup table enables the fast convergence of the gaze/focus to different targets, resulting in efficient continuous high-resolution observation of multiple dynamic objects. We experimentally verified the calibration accuracy, the high temporal response of gaze/focus during tracking onset, and the suitable identification performance for a very small moving marker under high-resolution tracking observation. We have also confirmed the long and detailed tracking of freely swimming fish under continuous observation.

Recently developed Single-photon Avalanche Diode (SPAD) array cameras have single photon sensitivity and can provide time-of-flight information for LIDAR imaging. These SPAD cameras, however, have very few pixels and readout binary images, which are typically averaged to provide an image with sufficient dynamic range. Here, we propose to implement a modified version of Fourier ptychography (FP), a synthetic aperture technique, on SPAD cameras to reconstruct an image with much higher resolution and larger dynamic range from its binary measurements. We successfully validate this using simulated and experimental results to show its potential for recording LIDAR images at high resolution and speed.

We demonstrate a deep-learning(DL)-based computational microscopy for high-throughput phase imaging by taking multiplexed measurements and employing deep neural networks (DNNs) based reconstruction. In particular, we develop a Bayesian convolutional neural network (BNN) to quantify the uncertainties of the DL inference, providing a surrogate estimate of the true prediction errors. The framework is demonstrated on a high-speed computational phase microscopy technique. We show the BNN is able to not only predict high-resolution phase images and but also provide a pixel-wise credibility map that evaluates the imperfections in the datasets and training process。