Electron microscope image stitching is highly desired to acquire microscopic resolution images of large target scenes in neuroscience. However, the result of multiple Mosaicked electron microscope images may exist severe gray scale inhomogeneity due to the instability of the electron microscope system and registration errors, which degrade the visual effect of the mosaicked EM images and aggravate the difficulty of follow-up treatment, such as automatic object recognition. Consequently, the grayscale correction method for multiple mosaicked electron microscope images is indispensable in these areas. Different from most previous grayscale correction methods, this paper designs a grayscale correction process for multiple EM images which tackles the difficulty of the multiple images monochrome correction and achieves the consistency of grayscale in the overlap regions. We adjust overall grayscale of the mosaicked images with the location and grayscale information of manual selected seed images, and then fuse local overlap regions between adjacent images using Poisson image editing. Experimental result demonstrates the effectiveness of our proposed method.
At present, deep learning is widely used and has achieved excellent results in many fields except in the field of image registration, the reasons are two-fold: Firstly all the steps of deep learning should be derivable; nevertheless, the nonlinear deformation which is usually used in registration algorithms is hard to be depicted by explicit function. Secondly, success of deep learning is based on a large amount of labeled data, this is problematic for the application in real scenes. To address these concerns, we propose an unsupervised network for image registration. In order to integrate registration process into deep learning, image deformation is achieved by resampling, which can make deformation step derivable. The network optimizes its parameters directly by minimizing the loss between registered image and reference image without ground truth. To further improve algorithm's accuracy and speed, we incorporate coarse-to-fine multi-scale iterative scheme. We apply our method to register microscopic section images of neuron tissue. Compared with highly fine-tuning method sift flow, our method achieves similar accuracy with much less time.
Proc. SPIE. 10574, Medical Imaging 2018: Image Processing
KEYWORDS: Image processing algorithms and systems, Image fusion, Cancer, Convolutional neural networks, Data modeling, Image segmentation, Electron microscopes, Scanning electron microscopy, Network architectures, Fusion energy
Recent studies have empowered that the relation between mitochondrial function and degenerative disorder- s is related to aging diseases. Due to the rapid development of electron microscope (EM), stacks delivered by EM can be used to investigate the link between mitochondrial function and physical structure. Whereas, one of the main challenges in mitochondria research is developing suitable segmentation algorithms to obtain the shapes of mitochondria. Nowadays, Deep Neural Network (DNN) has been widely applied in solving the neuron membrane segmentation problems in virtue of its exceptional performance. For this reason, its appli- cation to mitochondria segmentation holds great promise. In this paper, we propose an effective deep learning approach to realize mitochondria segmentation in Automated Tape-Collecting Ultra Microtome Scanning Elec- tron Microscopy (ATUM-SEM) stacks. The proposed algorithm contains three parts: (1) utilizing histogram equalization algorithm as image preprocessing to keep the consistency of dataset; (2) putting forward a fusion fully convolution network (FCN), which is motivated by the principle the deeper, the better, to build a much deeper network architecture for more accurate mitochondria segmentation; and (3) employing fully connected conditional random field (CRF) to optimize segmentation results. Evaluation was performed on a dataset of a stack of 31 slices from ATUM-SEM, with 20 images used for training and 11 images for testing. For comparison, U-Net approach was evaluated through the same dataset. Jaccard index between the automated segmentation and expert manual segmentations indicates that our method (90.1%) outperforms U-Net (87.9%) and has a preferable performance on mitochondria segmentation with different shapes and sizes.
Recently, due to the rapid development of electron microscope (EM) with its high resolution, stacks delivered by EM can be used to analyze a variety of components that are critical to understand brain function. Since synaptic study is essential in neurobiology and can be analyzed by EM stacks, the automated routines for reconstruction of synapses based on EM Images can become a very useful tool for analyzing large volumes of brain tissue and providing the ability to understand the mechanism of brain. In this article, we propose a novel automated method to realize 3D reconstruction of synapses for Automated Tapecollecting Ultra Microtome Scanning Electron Microscopy (ATUM-SEM) with deep learning. Being different from other reconstruction algorithms, which employ classifier to segment synaptic clefts directly. We utilize deep learning method and segmentation algorithm to obtain synaptic clefts as well as promote the accuracy of reconstruction. The proposed method contains five parts: (1) using modified Moving Least Square (MLS) deformation algorithm and Scale Invariant Feature Transform (SIFT) features to register adjacent sections, (2) adopting Faster Region Convolutional Neural Networks (Faster R-CNN) algorithm to detect synapses, (3) utilizing screening method which takes context cues of synapses into consideration to reduce the false positive rate, (4) combining a practical morphology algorithm with a suitable fitting function to segment synaptic clefts and optimize the shape of them, (5) applying the plugin in FIJI to show the final 3D visualization of synapses. Experimental results on ATUM-SEM images demonstrate the effectiveness of our proposed method.
Extracting the structure of single neurons is critical for understanding how they function within the neural circuits.
Recent developments in microscopy techniques, and the widely recognized need for openness and standardization
provide a community resource for automated reconstruction of dendritic and axonal morphology of single neurons. In
order to look into the fine structure of neurons, we use the Automated Tape-collecting Ultra Microtome Scanning
Electron Microscopy (ATUM-SEM) to get images sequence of serial sections of animal brain tissue that densely
packed with neurons. Different from other neuron reconstruction method, we propose a method that enhances the SEM
images by detecting the neuronal membranes with deep convolutional neural network (DCNN) and segments single
neurons by active contour with group shape similarity. We joint the segmentation and tracing together and they interact
with each other by alternate iteration that tracing aids the selection of candidate region patch for active contour
segmentation while the segmentation provides the neuron geometrical features which improve the robustness of tracing.
The tracing model mainly relies on the neuron geometrical features and is updated after neuron being segmented on
the every next section. Our method enables the reconstruction of neurons of the drosophila mushroom body which is
cut to serial sections and imaged under SEM. Our method provides an elementary step for the whole reconstruction of
Microscopic image registration is a key component of the neural structure reconstruction with serial sections of neural
tissue. The goal of microscopic neural image registration is to recover the 3D continuity and geometrical properties of
specimen. During image registration, various distortions need to be corrected, including image rotation, translation,
tissue deformation et.al, which come from the procedure of sample cutting, staining and imaging. Furthermore, there is
only certain similarity between adjacent sections, and the degree of similarity depends on local structure of the tissue and
the thickness of the sections. These factors make the microscopic neural image registration a challenging problem.
To tackle the difficulty of corresponding landmarks extraction, we introduce a novel image registration method for
Scanning Electron Microscopy (SEM) images of serial neural tissue sections based on the structure of mitochondria. The
ellipsoidal shape of mitochondria ensures that the same mitochondria has similar shape between adjacent sections, and its
characteristic of broad distribution in the neural tissue guarantees that landmarks based on the mitochondria distributed
widely in the image. The proposed image registration method contains three parts: landmarks extraction between
adjacent sections, corresponding landmarks matching and image deformation based on the correspondences. We
demonstrate the performance of our method with SEM images of drosophila brain.
Activity-dependent changes in the synaptic connections of the brain are tightly related to learning and memory. Previous studies have shown that essentially all new synaptic contacts were made by adding new partners to existing synaptic elements. To further explore synaptic dynamics in specific pathways, concurrent imaging of pre and postsynaptic structures in identified connections is required. Consequently, considerable attention has been paid for the automated detection of axonal boutons. Different from most previous methods proposed in vitro data, this paper considers a more practical case in vivo neuron images which can provide real time information and direct observation of the dynamics of a disease process in mouse. Additionally, we present an automated approach for detecting axonal boutons by starting with deconvolving the original images, then thresholding the enhanced images, and reserving the regions fulfilling a series of criteria. Experimental result in vivo two-photon imaging of mouse demonstrates the effectiveness of our proposed method.
In this paper we propose an image super-resolution algorithm based on Gaussian Mixture Model (GMM) and a new
adaptive image decomposition algorithm. The new image decomposition algorithm uses local extreme of image to
extract the cartoon and oscillating part of image. In this paper, we first decompose an image into oscillating and
piecewise smooth (cartoon) parts, then enlarge the cartoon part with interpolation. Because GMM accurately
characterizes the oscillating part, we specify it as the prior distribution and then formulate the image super-resolution
problem as a constrained optimization problem to acquire the enlarged texture part and finally we obtain a fine result.
This paper presents a wavelet-domain Hidden Markov Tree(HMT)-based color image superresolution algorithm using multi-channel data fusion. Because there exists correlations among the three channels of a RGB color image, a channel by channel superresolution method almost certain leads to color distortion. In order to solve this problem, first the low-resolution color image is converted into a gray-scale image using the spatially-adaptive approach presented in this paper and the resulting gray-scale image must reflect the human perception of edges in the color image; then by superresolving this gray-scale image, a high-resolution image is obtained; finally, wavelet-domain HMT-based image superresolutions are performed for the three channels of the low-resolution color image using the same posterior state probabilities, which reflect the hidden states of the wavelet coefficients of the high-resolution gray-scale image obtained before, and thus the resulting high-resolution color image is what we desired. Becasue the correlations among the three channels of a RGB color image are considered, there are no color distortions in the reconstructed high-resolution image. Experimental results show that the reconstructed color images have high PSNR and are of high visual quality.