24 September 2018 Label-free dynamic segmentation and morphological analysis of subcellular optical scatterers
Author Affiliations +
Imaging without fluorescent protein labels or dyes presents significant advantages for studying living cells without confounding staining artifacts and with minimal sample preparation. Here, we combine label-free optical scatter imaging with digital segmentation and processing to create dynamic subcellular masks, which highlight significantly scattering objects within the cells’ cytoplasms. The technique is tested by quantifying organelle morphology and redistribution during cell injury induced by calcium overload. Objects within the subcellular mask are first analyzed individually. We show that the objects’ aspect ratio and degree of orientation (“orientedness”) decrease in response to calcium overload, while they remain unchanged in untreated control cells. These changes are concurrent with mitochondrial fission and rounding observed by fluorescence, and are consistent with our previously published data demonstrating scattering changes associated with mitochondrial rounding during calcium injury. In addition, we show that the magnitude of the textural features associated with the spatial distribution of the masked objects’ orientedness values, changes by more than 30% in the calcium-treated cells compared with no change or changes of less than 10% in untreated controls, reflecting dynamic changes in the overall spatial distribution and arrangement of subcellular scatterers in response to injury. Taken together, our results suggest that our method successfully provides label-free morphological signatures associated with cellular injury. Thus, we propose that dynamically segmenting and analyzing the morphology and organizational patterns of subcellular scatterers as a function of time can be utilized to quantify changes in a given cellular condition or state.



Live cell imaging is crucial to understand the biological functions of a cell and its organelles. Fluorescent imaging has been a gold standard in this regard. While fluorescent stains provide good specificity and contrast, they also present some shortcomings including photobleaching and phototoxicity.1 In addition, cell transfection with fluorescent protein constructs, and sample staining, may complicate sample preparation or interfere with cell function and confound the results.2 To circumvent these shortcomings of imaging with fluorescent markers, label-free techniques, which do not require any stains or markers, are currently being developed.34.5

Here, we ultilize an optical scatter imaging (OSI) method based on dark-field microscopy and angular light scattering that is capable of tracking the distribution and morphology of subcellular structures, such as organelles, as a function of time. Imaging based on light scattering has previously enabled label-free detection and quantification of subcellular structural properties and dynamics pertaining to a biological process without the use of any exogenous marker.6 For example, wavelength and angularly-resolved scattering was used to extract nuclear size information.7,8 Wilson et al.9 used angularly resolved light-scattering measurements to detect mitochondrial swelling upon photodynamically induced oxidative stress. Light-scattering by various mitochondrial distributions has been studied with a finite-difference time-domain simulation, which was applied to differentiate between cancerous cells with randomly distributed mitochondria and healthy cells with aggregated mitochondria around the nucleus.10 In addition, we have previously utilized the ratio of wide-to-narrow angle scatter measurements to track calcium-induced mitochondrial injury11 and apoptosis.12 This previously reported OSI method employs optical Fourier filtering at a conjugate Fourier plane of the imaging system. More recently, via implementation of Gabor filters on the Fourier plane, we were able to probe objects of different size/shape and orientation.13,14 Based on this method, one morphometric parameter, termed orientedness, which probes the geometric aspect ratio of subcellular organelles was shown to be sensitive to the morphological changes related to mitochondrial fission during apoptosis in bovine aortic endothelial cells (BAEC).14

In this paper, we build upon our previous results by applying edge detection15 and segmentation16 methods to our filtered dark-field image data. Together, these methods allow us to amplify the signal from highly scattering subcellular structures relative to background, as well as segment these highly scattering microscopic regions within the cell cytoplasm. Combining these steps together, we develop an approach that can track subcellular dynamics noninvasively, in unstained cells over time. As a test of this method, we apply this approach to quantify morphological changes governed by mitochondrial fragmentation and remodeling induced by calcium overload. We demonstrate that the morphological signatures generated from unstained cells can clearly differentiate between injured and untreated cells.




Cell Culture

BAEC were cultured on glass coverslips as previously described.14 When comparing the dark-field data with fluorescence imaging, cells were incubated in Hepes buffered balanced salt solution11 supplemented with 100-nM MitoTracker Green (Invitrogen) for 45 min following the manufacturer’s protocol. The cover-slips containing the cells were mounted on a metal-slide for imaging as described in Ref. 14.


Calcium Treatment

To induce subcellular morphological change, the cells were overloaded with calcium via treatment with ionomycin (Sigma-Aldrich), a calcium ionophore. Pretreatment imaging was conducted in a salt-solution11 containing 1.5 mM CaCl2. The treatment solution was prepared by adding 20-μM ionomycin to the same saline medium. To induce the calcium overload, the chamber containing the cells was flushed three times with this treatment solution. Taking this point as t=0, time-lapse post-treatment images were collected every 45 s for a maximum of 10 min. For control, a 1.5 mM CaCl2 with no ionomycin was flushed three times instead of the treatment solution.


Optical Setup and Image Acquisition

Details of the optical setup are described in Ref. 13 and shown in Fig. 1. Briefly, an inverted microscope fitted with a 63×, N.A.=1.4 oil immersion objective is used to image the sample illuminated with a laser at λ=532  nm. The incident laser light is passed through a spinning diffuser to average speckle. Scattered light is collected from the sample in transmission mode and filtered by displaying images of Gabor filters on a liquid crystal display (LCD) placed in a conjugate Fourier plane (F2). To implement dark-field, the zeroth-order of diffraction is blocked by setting the corresponding pixels to 0 at the center of the LCD. The resulting filtered images are captured by a CCD camera placed in a conjugate imaging plane (I2). The OSI image magnification on the CCD at I2 is 0.25  μm/pixel. Through an additional port on the trinocular (not shown in the figure), the microscope also permits collection of epifluorescence images from the same sample on a different CCD camera.

Fig. 1

(a) OSI setup. The OSI technique involves the acquisition of transmission dark-field, filtered images on the charge-coupled device (CCD) camera. Two-dimensional (2-D) Gabor filters are displayed on a liquid crystal device (LCD) in the Fourier plane. L1 and L2: relay lenses, PBS: polarizing beam splitter, LP = linear polarizer, F1 and F2: conjugate Fourier planes, I1 and I2: conjugate imaging planes. The magnification on the CCD at I2 is 0.25  μm/pixel. (b) Illustration depicting a two-dimensional Gabor filter displayed on the LCD and positioned at coordinates (U,V) in Fourier space. Pixels at the center of the LCD are turned off and act as a zeroth order block to implement dark-field.



Gabor Filter Design

The Gabor filter images displayed on the LCD are created in MATLAB (The Mathworks, Natick, MA). In the object or image space, the filters are characterized by three parameters: period (S), orientation (φ), and standard deviation (σspace). While deviation determines the size of the object area probed by the Gabor filter, the period and standard deviation are typically chosen not to be independent16 so that the region probed by the filter scales with the filter’s period. For this work, we choose σspace=S2, and we choose six periods, S(inμm)=2.0, 1.43, 1.11, 0.91, 0.77, 0.66, and four orientations 90  degφ<225  deg with 45-deg increment. To implement the filters in a conjugate Fourier plane, we convert these parameters to their frequency-domain equivalents within the actual optical setup. The period transforms to the spatial frequency, F=1S and σfrequency=12πσspace.17 The filters are created in MATLAB by defining a Gaussian function as shown below. Derivation of the equation is shown in Table 1 (see Appendix):

where A is set to 255, and (U,V) are the coordinates in Fourier space and are calculated by

To calibrate for spatial frequency, a diffraction pattern with known spatial frequency spacing is used as described in Ref. 13. The calibration factor for the current setup is 0.0075  cycles/μm/LCD-pixel and the distance, F, of the Gaussian-peaks from the center of the LCD can be calculated using this factor. We also correct for the LCD’s reflectance by correcting the Gaussian’s grayscale values using the LCD’s measured look-up-table, which gives the LCD reflectance as a function of gray scale input. As such, the grayscale values inputted into the LCD will result in the expected Gaussian-shaped reflectance. Since the Fourier transform of a Gabor function is a center-shifted Gaussian, the filters are realized by placing the peaks of the Gaussians at a distance F from the center of the LCD. The coordinates of these pixels correspond to the spatial frequencies whose inverse corresponds to the prechosen periods.

Once the filtered-images are captured in the CCD, they are further processed, as shown in Fig. 2. To generate the orientedness images, local energy is calculated at each pixel first. Orientedness images are then masked using the dynamic masks generated using the maximum moment of phase-congruency (PC) images. Finally, features are extracted from both the binary masks and the masked-orientedness images. Calculation of local energy, orientedness, and PC is described in the following sections.

Fig. 2

Flowchart of the algorithm utilized for image analysis.



Generating Local Energy and Phase Congruency (Maximum Moment) Images

In one dimension, PC is defined as PC(x)=|E(x)|nAn(x), where An(x) are the amplitudes of n Fourier components at a location (x), and local energy E(x) is the vector summation of all components.15 PC acts as a highly localized operator to detect edges and corners in an image that is invariant to illumination and magnification. This technique takes advantage of the fact that Fourier components of the edge- or corner-points in an image are maximally in phase. This can be explained by a simple example of a square pulse. If all of the Fourier components of the square pulse are superimposed on each other, it can be shown that all of them will be in phase at the edges of the pulse. Thus, for the edge, the summation of the component-amplitudes is equal to the vector summation of all components, producing a PC value of 1. A minimum value of 0 may be obtained if all of the components are out of phase. Kovesi et al.15 proposed to extract the frequency information via wavelet transform instead of taking the Fourier transform. The wavelet transform is realized by even and odd symmetric log-Gabor filters of varying scale and orientation. Using the PC information, the edge-like features in the subcellular domain are then enhanced by taking the maximum moment of the PC covariance matrix. In this work, we have used an optimized code-version developed by Kovesi18 to generate the maximum moment and local energy images from each of the Gabor filtered images acquired in the setup. Thus, for each sample, 24 maximum moment images and 24 local energy images are calculated from the 24 optically filtered images acquired by the setup. These images are obtained from the “M” and “pcSum” variables found in the abovementioned code.

In addition, the user may set several input parameters pertaining to the calculation of PC. In particular, the local frequency information is obtained via digital log-Gabor wavelets of different scales and orientations. In this work, we set the number of scales (“nscale” in the code) to 4 and number of orientations (“norient”) to 4. We set the minimum wavelength of the wavelet (“minWaveLength”) to 2 pixels and the scaling between successive wavelengths (mult) to 2.1 Hence, for four scales, we obtain wavelets with wavelengths 2, 4.2, 8.82, and 18.5 pixels. To detect spatial properties at different orientations, four angles from 0 deg to 135 deg spaced 45 deg apart are used. The width of the filter function is controlled by the angular standard deviation σG of the function that is dependent on the filter center frequency, fo as R=σGfo. We set R=0.65 for this work (R corresponds to “sigmaOnf” in the code). Such combination of the scaling factor and R ensures an even coverage of the spectrum. Finally, we set “k” = 1, “cutoff” = 0.5 and “gamma” (gain) = 5. As described in Ref. 15, the code utilizes a weighting function that penalizes any PC value that is not spread beyond a certain frequency. This is because PC is significant only when it occurs over a wide range of frequencies. This function is of sigmoid form and can be controlled by two parameters: cutoff fraction, which dictates the amount of frequencies considered for the PC to be significant, and gain (gamma), which controls the steepness of the sigmoid function. In addition, “k” corresponds to the number of standard deviations of the noise energy beyond the mean at which the noise threshold is set. Only signal energies beyond this threshold will be considered. For noisy images, the value of “k” can be set up to 20. However, this will increase the threshold and result in loss of useful data. In our case, we set the value as low as 1 to avoid any such loss while discarding a minimum level of noise.


Generating Orientedness Images

Orientedness is a scattering-based parameter we had previously defined as the maximum signal over the average signal collected as a function of orientation, orientedness=max(Signal,φi)Signal,φi|S=0.9μm, and taken for data filtered with a Gabor period of 0.9  μm.14 φi is the orientation of the Gabor filters used in the experimental setup. For a round object, there will be an equal amount of scattering in all φ, resulting a minimum orientedness value of 1, whereas for an elongated object, the scatter profile will be oriented in a certain direction, producing an orientedness value greater than 1. As shown previously,14 orientedness decreases as long mitochondria fragment and round upon injury. In this work, we calculated orientedness using the local energy images, defined in the previous section, instead of the raw intensity values {signal,φi}. Here, 90  degφi<225  deg, and the optically filtered images with a Gabor filter period of 2  μm were used in this calculation. Hence, the modified equation is given by orientedness=max(local energy,φi)local energy,φi|S=2μm.


Image Segmentation

First, the nucleus regions as well as the background outside the cells (no-cell region) are segmented out manually from the cell images so as to only analyze the scatterers within the cytoplasm. Then, following the main frame-work proposed by Jain and Farrokhnia,16 each Gabor-filtered image is preprocessed and fed to a clustering algorithm. However, we applied PC as a preprocessing step instead of the “blob-detection” approach applied in Ref. 16. Then, pixels in the maximum moment images obtained above were subjected to a K-means clustering algorithm. K-means clustering is an unsupervised learning approach that is used to assign data to a number of groups, K, based on the distances of data-points from K initially assigned centroids. The iterative algorithm assigns each data-point to one of the K groups until all of the points are assigned.19 After each iteration, each of the K centroids is moved to the median intensity value of the pixels in its group. The algorithm is repeated until the sum of the point-to-centroid distances over all clusters is minimized. For this, we let the algorithm assign the initial centroids. We used the “cityblock” distance algorithm in MATLAB for the K-means distance calculation. We took the median value instead of the mean to avoid the effect of any outlier. Pixels in the 24 optically filtered images are grouped into four clusters here. We found 4 to be the highest number of clusters to be used for these images as the algorithm does not converge for a greater number of clusters. Pixels categorized in each cluster are color-labeled with one of the four colors: blue, turquoise, yellow, and red. In this analysis, all the experimental timepoints for a given cell were analyzed at once, yielding four classes evolving as a function of time.




Cell Images Produced by the Algorithm

Images obtained at different steps are shown in Fig. 3. The Gabor-filtered image is obtained with S=2.0  μm, φ=225  deg, and the corresponding PC (maximum moment), local-energy, and orientedness images are shown. The dark-field image is also shown. The dark-field image contains all spatial-frequency components; hence, it contains the scattering information from objects of all sizes and orientations. On the other hand, it is mostly the objects oriented at 225 deg that are highlighted in the Gabor-filtered image. Enhanced edges of these objects can be seen in the maximum moment image. The orientedness image maps the orientedness values of the subcellular objects in the cytoplasm. As indicated by the color scale, objects with a hue toward the blue have low orientedness and hence are round compared to the objects shown in red.

Fig. 3

(a) Dark-field, (b) Gabor-filtered (S=2.0, φ=225  deg), (c) PC (maximum moment), (d) local energy, and (e) orientedness images of a representative cell. Note that edges are enhanced in the PC (maximum moment) image.



Tracking Subcellular Organelles with Dynamic Masking

To segment the subcellular objects in the cytoplasm, we applied a K-means algorithm to classify the pixels in the maximum moment images. Before classification, the data are standardized by subtracting the mean and dividing by the standard deviation to obtain zero mean and unit variance. Figure 4 shows the four colored pixel clusters resulting from the K-means algorithm for a representative cell. To verify whether the clusters contain pixels with significant morphological information as well as to discard clusters that contain pixels from the background, we analyze the four centers, or centroids, of the four clusters [Fig. 5(a)]. Each centroid has 24 centroid components, representative of the 24 filtered intensities of pixels within the cluster associated with that centroid. The 24 centroid values are arranged as a function of optical filter period, S, and orientation φ. As can be seen in the polar-plots [Fig. 5(a)], the centroid values of the turquoise (cluster 2) and red labels (cluster 4) vary as a function of period and orientation. On the other hand, there is no or little change in the centroid values for blue (cluster 1) and yellow labels (cluster 3). The blue and yellow clusters also have lower values. This indicates that the pixels from these two clusters are either from the background outside the cell or the nuclei, which were manually segmented out (blue in Fig. 4), or subcellular regions with significantly less intensity than the red and turquoise regions (yellow in Fig. 4). We quantified the variation in cluster values by taking the magnitude (mag) of each cluster, where mag=S,φc2 and c is the centroid-value corresponding to a filter (S,φ). The bigger the magnitude, the greater the variation over period and orientation. We then pick two clusters with the highest magnitudes and add them to create a single binary mask for the pretreatment condition [Fig. 5(b)]. Keeping the selected clusters the same, we repeat the combining process for each time point of data acquisition to generate masks that change over the period of time. We then create “masked orientednedness” images by multiplying the orientedness images with the binary masks at each time point (Fig. 6).

Fig. 4

Four clusters of pixels labeled in blue, turquoise, yellow, and red. Top panels: pretreatment, and bottom panels: post-treatment. Each of these clusters contains objects with different scattering profiles and intensities. Note also how the shape of the labeled objects changes with treatment; particularly elongated objects in the turquoise-cluster become smaller and rounder.


Fig. 5

(a) Color-coded centroid values are arranged in a 4×6 polar plot corresponding to four orientations and six periods for each label shown in Fig 4. In both pre- and post-treatment cases, the centroid components of the turquoise and red clusters’ demonstrate variation over orientations (φ) as well as periods (S), while those of the blue and yellow clusters have low signal with little or no variation. (b) The turquoise and red clusters are added together to produce a combined final binary mask.


Fig. 6

Masked orientedness image of a representative cell. Gradual change in the subcellular scatterers and shape of the dynamic mask can be observed upon injury at t=0.


As can be seen in Fig. 6, dynamic masks created by the proposed algorithm change shape over time as the cell shrinks upon severe chemical insult (Fig. 7). Most of the long objects in the masks become fragmented or round along with the cell-shrinkage and exhibit a decrease in their orientedness value. Concurrently, mitochondria become shorter and rounder upon calcium-overloading, observed in the MitoTracker-labeled fluorescent images (Fig. 8).

Fig. 7

Differential interference contrast (DIC) image of a cell under calcium-overload. (a) Long objects (yellow arrow) observed in the pretreatment image fragment and (b) become round due to excessive calcium. The cell also shrinks which is indicated by trace-marks of the cell membrane (pink arrow-head). When compared with MitoTracker-labeled fluorescent images, the fragmented and rounded long objects in the DIC images correspond to mitochondria (Fig. 8).


Fig. 8

MitoTracker labeled fluorescent images show mitochondrial fragmentation and remodeling upon calcium-overload.



Effect of Jain’s Multichannel Filtering Approach

Jain’s framework is based on a multichannel filtering approach, originally proposed in Ref. 20, which dictates that the human visual system decomposes retinal information into a number of filtered images. The “multichannels” are realized by the Gabor filters. To understand the effect of Gabor filters on the final segmented image with classified pixels, we applied PC and clustering of the pixels on the unfiltered dark-field image. Segmented images obtained using the single-channel (DF only) and multichannel (Gabor) contain almost the same information in the cellular region, except the former has noise (Fig. 9). The multichannel approach decomposes the same information of the dark-field into multiple channels and allows selecting the channels less corrupted by noise. Hence, we can discard the channels manually (i.e., Gabor filters of particular period and orientation) that are corrupted by noise. This enabled us to generate a clearer image compared to that obtained by using dark-field only.

Fig. 9

(a) Segmented images generated using the multichannel Gabor-filtered images and (b) the single unfiltered dark-field image.



Aspect-Ratio and Orientedness Decrease upon Injury

To quantify the subcellular morphological changes, we analyzed the subcellular objects segmented by the image masks obtained above (Fig. 5). Objects are defined as clusters of pixels with connectivity with four pixels.21 We measured the aspect-ratio of the objects over time to quantify the subcellular morphological changes [Fig. 10(a)]. We also calculated the median orientedness from the pixels contained in each object. Figure 10(b) shows that the object’s median orientedness decreases upon cell injury. Analysis of the aspect ratio of the MitoTracker labeled objects also shows a simultaneous decrease with treatment [Fig. 10(c)].

Fig. 10

Segmented subcellular objects become rounder after calcium overload, as indicated by the (a) aspect-ratio and (b) orientedness. The decrease in the aspect ratio of objects within the fluorescence images (c) suggests that the change in the aspect ratio of mitochondria partially accounts for the decrease in the masked subcellular objects’ aspect ratio. The data show mean and standard error for n=6 cells for the first six timepoints (treated) and the first seven timepoints (control). Additional timepoints were tested for one control cell and four of the treated cells.



Texture-Features Can Detect Subcellular Morphological Dynamics Induced by Injury

Calcium-injury perturbs the spatial arrangement of the subcellular organelles. Such rearrangement is a result of changes in organelle morphology, as well as cell-shrinkage. While the treatment produces objects with a decreased aspect ratio and orientedness [Figs. 10(a) and 10(b)], relative distances among organelles also change and reduce as the cell shrinks. To quantify these two-dimensional, cell-wide, structural changes associated with the injury, we analyzed the texture of the masked orientedness images as a function of time. To this end, we extracted eight texture-features2223.24 using the code available25 to analyze the spatial variation in the orientedness values. As proposed in Ref. 24, we grouped the features into three categories; features that describe the smoothness [contrast, dissimilarity, and inverse difference moment (IDM) or homogeneity], uniformity (entropy, maximum probability, and energy) and correlation (autocorrelation, correlation) of the texture.

We found that multiple texture features are sensitive to the morphological changes encoded by the orientedness images. We plotted these texture-features as a heat-map (Fig. 11) for each cell and observed the change in color over time, which indicates the features’ ability to detect the underlying structural changes. Several features changed by more than 30% in the treated cells. Autocorrelation, contrast, dissimilarity, and entropy increased in the treated cells, while they decreased slightly or remained within 10% of the starting value in the control cells. Energy and maximum probability decreased in the treated cells but remained within 10% of the starting value in the control cells. Cell-to-cell variations are also observed in both the control and treated cells. We also observed individual feature-responses for the whole dataset (Fig. 12). Autocorrelation, correlation, contrast, dissimilarity, and entropy increase while energy, maximum probability, and IDM decrease over time in the treated cells compared with the control. We select autocorrelation, contrast, entropy as the representative features from each group with positive trends (i.e., increase) and energy and IDM as negative trends (i.e., decrease). We then calculate the composite feature by subtracting the decreasing features from the increasing ones. The square root of the energy feature is taken to match the unit with that of other features. The composite feature in Fig. 13 also shows a shift from the baseline after injury.

Fig. 11

Changes in the texture-features of the masked orientedness images shown as heat-maps. Each row is a feature while each column is a time-point. Cells were monitored for different time periods, but for at least 180 s, after ionomycin treatment (t=0, treated), or the same saline solution but with no ionomycin (t=0, control). Texture-features are sensitive to the change in the spatial arrangement of the morphometric parameter orientedness. The color scale represents the magnitude of each parameter.


Fig. 12

Average responses of the individual texture features over time. The smoothness of the texture decreases as the contrast and dissimilarity increase and IDM (also known as homogeneity) decreases in the treated cells compared with controls. The uniformity of the texture decreases as the entropy increases, whereas the energy and max. probability decrease. Correlation between pixels increases. The data show mean and standard error for n=6 cells for the first six timepoints (treated) and the first seven timepoints (control). Additional timepoints were tested for one control cell and four of the treated cells. (a) Autocorrelation, (b) correlation, (c) contrast, (d) dissimilarity, (e) entropy, (f) energy, (g) maximum probability, and (h) IDM.


Fig. 13

Composite feature calculated from representative features: autocorrelation, contrast, entropy, energy, and IDM. The data show mean and standard error for n=6 cells for the first six timepoints (treated) and the first seven timepoints (control). Additional timepoints were tested for one control cell and four of the treated cells.




Cells and their subcellular organelles undergo morphological changes constantly. These changes occur either naturally or upon treatment. To quantify such morphological changes in an unstained cell, we utilized OSI to detect the scatterers present within the cell. However, tracking these scatterers and quantifying their morphology over time is a challenge since they are constantly shifting in space and changing shape. To address this problem, we present an algorithm to process angular-scatter-encoded images to detect as well as to track structural changes in the subcellular domain. With this algorithm, we generate a set of masks, which are used to segment individual subcellular scatterers, and dynamically change shape in response to the morphological changes in the scatterers over time (Figs. 5 and 6). These masks further allow us to extract morphometric parameters, such as the aspect ratio of the segmented objects (Fig. 10) or analyze the textural features of the mask-images (Figs. 1113).

To test this algorithm, we induced morphological changes by overloading endothelial cells with calcium. Excess calcium leads to the opening of the mitochondrial permeability transition (mPT) pores. Subsequent redistribution of small molecules and water causes swelling in the mitochondria.26 Due to calcium overload, a long mitochondrion may either remodel or undergo fission. Mitochondria undergoing remodeling become round, whereas fission produces smaller fragments.27 With a sufficiently high calcium dose as used here, cells will also shrink and undergo cell death. As a result of this treatment, the organelles’ morphology changes as well as their subcellular structural arrangement as the cell starts to shrink and subsequently die. This calcium treatment is similar to the one published in Ref. 11. Thus, one significant aspect of this treatment is that it results in mitochondrial fragmentation and rounding. As we had previously observed, mitochondria in this study also attain a smaller and rounder shape postinjury as opposed to the elongated shape pretreatment (Fig. 8). Given the twofold transformation in the morphology and arrangement of subcellular organelles upon treatment, we apply our algorithm and dynamic masks to track the scatterers over time and quantify their morphological changes. Our results show that the aspect-ratio and orientedness of the segmented objects in the masks decrease [Figs. 10(a) and 10(b)]. This suggests that these masks are sensitive to the rounding of the organelles. This change was concurrent with a decrease in the aspect ratio of MitoTracker-labeled mitochondria [Fig. 10(c)], suggesting that the decrease in the scattering objects’ aspect ratio may be at least partially accounted for by the rounding of the mitochondria becoming shorter and rounder upon calcium overloading.

We had used the “orientedness” parameter previously to measure mitochondrial fission.14 In that work, average orientedness values were calculated for each cell based on raw optical filtered intensities. The orientedness parameter takes advantage of the fact that objects of different geometries (e.g., noncircular versus circular) have different directional light-scattering profiles. For example, light scattered from an elongated object is confined in a smaller azimuth angle, φ in the scatter-plane as compared to that of a round object.11 Orientedness measures this anisotropy of scattered light by taking the ratio of light-scatter signal at the direction of maximum scatter to the mean signal collected at all available directions.14 This ratio is measured at each pixel, resulting in a cell-wide orientation mapping. However, since orientedness is a normalized ratio of two intensity values, it is independent of signal strength at a given pixel, and pixels in the background may show orientation, which does not have any biological relevance. Hence, to amplify signal from the subcellular organelles as well as to suppress signal from background, we present a modification of the orientedness parameter that uses local energy information, instead of only raw intensity values. Local energy emphasizes regions of abrupt change in intensity such as edges in an image. We take this property of local energy to enhance the edges of the organelles as well as to suppress signal from monotonically varying intensity such as the background. By multiplying the orientedness images with the masks, we further extract signal from the major scattering sources, allowing us to analyze individual subcellular scatterers over time, as opposed to taking the average orientedness across the whole cell. When tracked over time, we observe a decrease in the orientedness of the objects of the masks. While the aspect-ratio measures an object’s shape directly by taking the ratio of the height and weight, orientedness provides an indirect means to obtain similar information from the scattering profile of the object in question. As expected, when tracked over time, both of these parameters show a similar decreasing trend with time [Figs. 10(a) and 10(b)], indicating concordance between two approaches. Correlation between the fluorescent data [Fig. 10(c)] and data obtained from the label-free approach suggests that the morphological change probed by our optical scatter technique is pertinent to mitochondria. These results corroborate our previous data showing detection of mitochondrial rounding and fission with optical scattering.11,14 However, the present data improve on our previous parameters. Here, we generate a dynamic mask that can change shape over time to extract orientedness values only from the scatterers within the cell and discard background pixels. Thus, our current data are less sensitive to background and can enable tracking of individual scatterers.

To generate our subcellular organelle masks, we apply a segmentation technique originally proposed for digital Gabor-filtered images16 and multiply the resulting segmentation mask with the orientedness image. The masked-orientedness images therefore contain orientation information from significant scatterers in the cell. As the organelles change shape, the transformation of their orientedness values will impact the spatial relationship between two pixels. In addition, the spatial arrangement of the organelles will also be affected by the shrinkage of the cell upon injury. While the change in the individual masked objects reflects changes in individual structures, the spatial distribution and arrangement of these structures may be quantified using various image-based texture-features. Our texture analysis shows that autocorrelation, correlation, contrast, dissimilarity, and entropy increase while energy, maximum probability, and IDM decrease over time (Fig. 12). However, as suggested by Haralick et al.,22 even though these features show sensitivity toward change in texture, it is difficult to identify which textural characteristic is signified by each of them. Hence, it will be hard to draw any biological relevance of the extracted features. Thus, one way to represent the underlying biological events is to combine these features and observe their ensemble behavior in one composite textural parameter (Fig. 13). Taken together, our data allow for single-scatterer analysis via the analysis of the individual masked objects, and whole-cell analysis via the textural features. The textural features are indicative of an overall cellular dynamic morphological state that changes in response to treatment.

Traditionally, organelles in live cells are visualized either by staining with dyes or by expressing organelle-targeted fluorescent proteins. However, such approaches can have adverse effects on organelle function. In particular, fluorescent labels have adverse effects on mitochondrial metabolism1 and dynamics,2 which can ultimately hinder reliable drug screening. For example, MitoTracker dyes reduce cell motility in primary neuronal culture as well as alter their morphology.2 Besides, some dyes are not photostable if used in low concentration to avoid quenching effects, whereas fluorophores tend to aggregate and stain other organelles if used in high concentration.28 On the other hand, mitochondria targeted fluorescent proteins such as mito-GFP do not affect function or morphology; however, long-duration time-lapse imaging is not possible due to phototoxicity.1 Moreover, low transfection efficiency hinders high-throughput screening for drug discovery.29 Hence, a label-free imaging technique would be ideal for noninvasive quantification of organelle dynamics over a long period. To this end, several studies have recently demonstrated the potential of label-free methods used in conjunction with computational image processing for cellular analysis and classification. For example, Blasi et al.3 proposed a label-free approach to detect DNA content and quantify mitotic cell cycle phases by combining conventional flow cytometry and single-cell imaging. The phases were classified using morphological features extracted from bright-field and dark-field images. In this paper, we use similar features, such as aspect-ratio and Haralick textures. However, one major difference here is the use of the orientedness parameter, which is based on light-scattering. Since light-scattering is directly related to the size and shape of the probed objects, the orientedness parameter contains valuable information regarding morphological changes in the subcellular organelles.

One of the limitations of the technique presented here is its dependency on image-processing parameters that generate local energy and PC images. However, once these parameters are tuned empirically for a specific cell type, they are constant for any number of samples. This procedure will be automated in the future. The proposed features can also be used in conjunction with other dynamic parameters, such as velocity and displacement of the organelles in biologically relevant cases, such as mitochondrial mobility, fission/fusion rate, etc. The combined approach can then be used to provide a better label-free quantification of subcellular dynamics.

In conclusion, we have developed a label-free approach to detect and track subcellular morphological changes dynamically. This technique does not use any exogenous marker and hence can be used to avoid the adverse effects of label-based assays and potentially provide a method for rapid drug-screening. In the future, we hope to apply this technique to different mammalian cell types, such as neurons, to quantify subcellular dynamics upon chemically and mechanically induced injuries.



Implementation of Gabor filters on the conjugate Fourier plane requires conversion of the related parameters from space to frequency domain. This conversion is detailed in Table 1.

Table 1

Expression for Gabor filters in frequency domain.

General expression of a Gaussian function: H(u,v)=A*e[(uU)2+(vV)22σfrequency2]; σfrequency=12πσspace; with σspace=S2.
Hence, σfrequency=1πS
Replacing σfrequency in the general expression, we obtain: H(u,v)=A*e[(uU)2+(vV)22/π2S2], where period, S=1F=1U2+V2


No conflicts of interest, financial or otherwise, are declared by the authors.


This paper is based in part on the SPIE proceedings paper titled “Label-free Fourier filtered dark-field imaging to quantify subcellular dynamics,” by Mohammad Naser, Rene S. Schloss, and Nada N. Boustany, Proc. SPIE 10504, Biophysics, Biology and Biophotonics III: the Crossroads, Adam Wax; Vadim Backman, Editor(s), 105040G (20 Feb. 2018); doi:  https://doi.org/10.1117/12.2290908. This work was partially supported by grant CBIR14PIL005 from the New Jersey Commission on Brain Injury Research and NSF under Grant No. CBET1512170. We thank Dr. Matthieu Petitjean of Pharmanest, LLC (Princeton, New Jersey) for insightful discussions regarding the analysis of the masked subcellular objects.


1. R. Dixit and R. Cyr, “Cell damage and reactive oxygen species production induced by fluorescence microscopy: effect on mitosis and guidelines for non-invasive fluorescence microscopy,” Plant J. 36(2), 280–290 (2003).CJRSDP0703-8992 https://doi.org/10.1046/j.1365-313X.2003.01868.x Google Scholar

2. J. F. Buckman et al., “MitoTracker labeling in primary neuronal and astrocytic cultures: influence of mitochondrial membrane potential and oxidants,” J. Neurosci. Methods 104(2), 165–176 (2001).JNMEDT0165-0270 https://doi.org/10.1016/S0165-0270(00)00340-X Google Scholar

3. T. Blasi et al., “Label-free cell cycle analysis for high-throughput imaging flow cytometry,” Nat. Commun. 7, 10256 (2016).NCAOBW2041-1723 https://doi.org/10.1038/ncomms10256 Google Scholar

4. M. C. Skala et al., “In vivo multiphoton microscopy of NADH and FAD redox states, fluorescence lifetimes, and cellular morphology in precancerous epithelia,” Proc. Natl. Acad. Sci. U. S. A. 104(49), 19494–19499 (2007). https://doi.org/10.1073/pnas.0708425104 Google Scholar

5. C. Mujat et al., “Endogenous optical biomarkers of normal and human papillomavirus immortalized epithelial cells,” Int. J. Cancer 122(2), 363–371 (2008).IJCNAW1097-0215 https://doi.org/10.1002/ijc.23120 Google Scholar

6. N. N. Boustany, S. A. Boppart and V. Backman, “Microscopic imaging and spectroscopy with scattered light,” Annu. Rev. Biomed. Eng. 12, 285–314 (2010).ARBEF71523-9829 https://doi.org/10.1146/annurev-bioeng-061008-124811 Google Scholar

7. L. T. Perelman et al., “Observation of periodic fine structure in reflectance from biological tissue: a new technique for measuring nuclear size distribution,” Phys. Rev. Lett. 80(3), 627–630 (1998).PRLTAO0031-9007 https://doi.org/10.1103/PhysRevLett.80.627 Google Scholar

8. A. Wax et al., “Cellular organization and substructure measured using angle-resolved low-coherence interferometry,” Biophys. J. 82(4), 2256–2264 (2002).BIOJAU0006-3495 https://doi.org/10.1016/S0006-3495(02)75571-9 Google Scholar

9. J. D. Wilson et al., “Light scattering from intact cells reports oxidative-stress-induced mitochondrial swelling,” Biophys. J. 88(4), 2929–2938 (2005).BIOJAU0006-3495 https://doi.org/10.1529/biophysj.104.054528 Google Scholar

10. X. T. Su et al., “Light scattering characterization of mitochondrial aggregation in single cells,” Opt. Express 17(16), 13381–13388 (2009).OPEXFF1094-4087 https://doi.org/10.1364/OE.17.013381 Google Scholar

11. N. N. Boustany, R. Drezek and N. V. Thakor, “Calcium-induced alterations in mitochondrial morphology quantified in situ with optical scatter imaging,” Biophys. J. 83(3), 1691–1700 (2002).BIOJAU0006-3495 https://doi.org/10.1016/S0006-3495(02)73937-4 Google Scholar

12. N. N. Boustany et al., “BCL-xL-dependent light scattering by apoptotic cells,” Biophys. J. 87, 4163–4171 (2004).BIOJAU0006-3495 https://doi.org/10.1529/biophysj.104.048736 Google Scholar

13. H. Sierra et al., “Measurement of object structure from size-encoded images generated by optically-implemented Gabor filters,” Opt. Express 20(27), 28698–28706 (2012).OPEXFF1094-4087 https://doi.org/10.1364/OE.20.028698 Google Scholar

14. R. M. Pasternack, J. Y. Zheng and N. N. Boustany, “Detection of mitochondrial fission with orientation-dependent optical Fourier filters,” Cytometry A 79(2), 137–148 (2011). https://doi.org/10.1002/cyto.a.v79a.2 Google Scholar

15. P. Kovesi et al., “Image features from phase congruency,” Videre: J. Comput. Vis. Res. 1(3), 1–26 (1999). Google Scholar

16. A. K. Jain and F. Farrokhnia, “Unsupervised texture segmentation using Gabor filters,” Pattern Recognit. 24(12), 1167–1186 (1991). https://doi.org/10.1016/0031-3203(91)90143-S Google Scholar

17. J. W. Goodman, Introduction to Fourier optics, 3rd ed., Roberts and Company Publishers, Englewood, Colorado (2005). Google Scholar

18. P. Kovesi, “MATLAB and Octave functions for computer vision and image processing,”  http://www.peterkovesi.com/matlabfns/PhaseCongruency/phasecong3.m (7 March 2018). Google Scholar

19. J. T. Tou, “Pattern recognition principle,” Appl. Math. Comput. 7, 75–109 (1974). Google Scholar

20. F. W. Campbell and J. G. Robson, “Application of Fourier analysis to the visibility of gratings,” J. Physiol. 197(3), 551–566 (1968).JPHYA70022-3751 https://doi.org/10.1113/jphysiol.1968.sp008574 Google Scholar

21. R. C. Gonzalez and R. E. Woods, Chapter 9 in Digital Image Processing, 2nd ed., Prentice Hall, Boston, Massachusetts (2007). Google Scholar

22. R. M. Haralick, K. Shanmugam and I. Dinstein, “Textural features of image classification,” IEEE Trans. Syst. Man Cybern. 3(6), 610–621 (1973). https://doi.org/10.1109/TSMC.1973.4309314 Google Scholar

23. L. K. Soh and C. Tsatsoulis, “Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices,” IEEE Trans. Geosci. Remote Sens. 37(2), 780–795 (1999).IGRSD20196-2892 https://doi.org/10.1109/36.752194 Google Scholar

24. D. A. Clausi, “An analysis of co-occurrence texture statistics as a function of grey level quantization,” Can. J. Remote Sens. 28(1), 45–62 (2002). https://doi.org/10.5589/m02-004 Google Scholar

25. P. Brynolfsson, “GLCMFeatures(glcm),” Version,  https://www.mathworks.com/matlabcentral/fileexchange/55034-glcmfeatures-glcm (29 August 2018). Google Scholar

26. R. Jemmerson, J. M. Dubinsky and N. Brustovetsky, “Cytochrome C release from CNS mitochondria and potential for clinical intervention in apoptosis-mediated CNS diseases,” Antioxid. Redox Signal. 7(9–10), 1158–1172 (2005). https://doi.org/10.1089/ars.2005.7.1158 Google Scholar

27. S. Deheshi et al., “Changes in mitochondrial morphology induced by calcium or rotenone in primary astrocytes occur predominantly through ROS-mediated remodeling,” J. Neurochem. 133(5), 684–699 (2015).JONRA90022-3042 https://doi.org/10.1111/jnc.2015.133.issue-5 Google Scholar

28. Q. Zhao, C. Huang and F. Li, “Phosphorescent heavy-metal complexes for bioimaging,” Chem. Soc. Rev. 40(5) 2508–2524 (2011).CSRVBR0306-0012 https://doi.org/10.1039/c0cs00114g Google Scholar

29. Z. Zhang et al., “Quality control of cell-based high-throughput drug screening,” Acta Pharm. Sin. B 2(5) 429–438 (2012). https://doi.org/10.1016/j.apsb.2012.03.006 Google Scholar


Mohammad Naser is a PhD candidate in biomedical engineering at Rutgers University. After completing his master’s degree in electrical and electronic engineering from Bangladesh, he joined Dr. Nada Boustany’s Bio-Optics Lab in 2013 and he has been working to develop a Fourier-Optics based label-free imaging technique. His research interest includes optics, biology, image and signal processing, pattern recognition, and machine learning.

Rene S. Schloss is an assistant research professor in biomedical engineering at Rutgers University. She received her PhD in immunology from Harvard University and she did postdoctoral work at Sloan Kettering in Molecular Immunology. Her current research interests include cellular and tissue engineering.

Pauline Berjaud: Biography is not available.

Nada N. Boustany is an associate professor of biomedical engineering at Rutgers University. Her lab develops quantitative imaging methods to measure and analyze mitochondrial structure and function with light scattering. She is interested in label-free techniques that can characterize subcellular structural dynamics pertaining to apoptosis and cell injury. She received her PhD from the Harvard-MIT Division of Health Sciences and Technology, and conducted postdoctoral work in bioengineering at Johns Hopkins University.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Mohammad Naser, Mohammad Naser, Rene S. Schloss, Rene S. Schloss, Pauline Berjaud, Pauline Berjaud, Nada N. Boustany, Nada N. Boustany, "Label-free dynamic segmentation and morphological analysis of subcellular optical scatterers," Journal of Biomedical Optics 23(9), 096004 (24 September 2018). https://doi.org/10.1117/1.JBO.23.9.096004 . Submission: Received: 4 April 2018; Accepted: 13 August 2018
Received: 4 April 2018; Accepted: 13 August 2018; Published: 24 September 2018

Back to Top