1.

Introduction

Very often one cannot recover recognizable images from subdiffraction-limit sized particles by optical imaging due to scattering. Using the Fourier spatial frequency (FSF) spectrum, valuable information from the scattered light from aperiodic and random structures formed by such particles can be recovered. It is well established that the intensity pattern in an image plane is a composition of “spatial frequencies” much in the same way that a spectral domain signal is composed of various color frequencies. The spatial frequency spectrum can be directly obtained from the transmission electron microscope (TEM) and optical microscope images by taking the Fourier transform (FT); each image containing a waveform composed of spatial frequencies, with units of cycles per unit distance, which is analogous to the Fourier decomposition of color frequencies—in cycles per cm—used in frequency-domain analysis.¹ Laser speckle analysis is one method that uses spatial frequencies to distinguish regions of moving material from motionless material. These regions are then stitched together in laser speckle imaging (LSI) to reveal detailed images,² or analyzed in the frequency domain in micro-optical coherence tomography.³ Cerbino and Trappe⁴ introduced a related novel technique called Differential Dynamic Microscopy (DDM) in which they used Fourier optics to investigate the time-resolved dispersion of 73- and 420-nm diameter colloidal particles in aqueous suspension. Each of these methods derive their constitutive equations from the mathematical formalism of the FT of scattered light in an image plane, and then integrate over the region of interest to obtain a measure of correlation as a function of total intensity. Most recently Pu et al. used FSF analysis to detect early stages in cervical cancer. Using confocal fluorescence images of H&E stained stroma, they focused on the identification of characteristic spatial-spectrum signatures to distinguish between normal and neoplastic tissue.⁵

The fluid mechanics of tumor interstitial fluid (TIF) are difficult to evaluate. It is known that interstitial pressure in the tumor deviates significantly from that in normal tissue, typically developing a high internal pressure and a strong pressure gradient from the interior of the tumor outward.⁶ This in turn dictates that interstitial fluid flow in tumor will deviate significantly from that in normal tissue. But the technological challenges to detect and monitor the TIF pressure are so great that only a limited number quantitative studies have ever been attempted in the area,⁷ and experiments to detect and quantify convective flow of the TIF through tumor have yet to be reported. Molecular targeting strategies have been rigorously investigated in recent years, and provide a detailed map of where targeted agents go, allowing us to indentify to some extent the flow pathways within the tumor.^8,9 Unfortunately we cannot identify barriers to transport except by evidence of failure.

It is evident that the heterogeneous structure and composition of the extracellular matrix (ECM) has a crucial role to play in intra-tumor flow.¹⁰ The composition of the TIF is such that microstructural order can develop due to particle interaction, leading in turn to high particle pressure in shear flow, which is related to shear thickening and jamming.^11,12 The difficulties of predicting the transport of an exogenous agent within the TIF emerges even in pharmacokinetic models, which can predict the effect of agents delivered to the plasma compartment, but fail to predict the outcome in solid tumor.¹³ To further complicate matters, sustained interparticle forces may lead to an irreversible aggregation. Under certain conditions, native lipoproteins, a principal component of the TIF, have been observed to develop a sealing effect that retards trans-endothelial transport.¹⁴ Unfortunately, we have no means to observe these interactions and structures directly. In vivo methods have a limit of detection in the range of ${10}^{-5}\mathrm{m}$, and the diffraction limit of experimentally available light, on the order of 200 nm, effectively sets a lower bound on all optical methods.

This letter describes an innovative approach to use spatial frequency analysis of stationery TEM images to characterize the self assembled structures of high-density lipoproteins (HDL) in aqueous solution, and to distinguish the different length scales of particle interaction at each stage of aggregation, from single-particle to an array. Information on the HDL structure in tissue-like media is critical for understanding the effect of HDL structure on transport of anti tumor drugs within the tumor.

Figure 1 shows the image of TEM of reconstituted high-density lipoprotein (rHDL) particles; from single particles suspension [Fig. 1(a)], to jammed solid [Fig. 1(b)], to fused membrane [Fig. 1(c)]. Understanding the distribution of superparamagnetic iron-oxide nanoparticles (SPION) within target tissues is vital to the application of these important contrast agents in imaging cancer and atherosclerosis. Since, the SPION have been shown to interact with the ECM and the TIF,¹⁵ we have prepared a typical formulation, admixed with rHDL at a $1:100$ ratio. Our objective is to extract the characteristic spatial frequencies of a given sample, so that it can be uniquely identified by a spatial frequency spectrum taken of any image of the sample, and quantitatively compared for similarity to the spatial frequency spectrum of other images of similar or different samples. The overarching goal of this work is to develop a method for pattern recognition to allow characterization of the TIF microstructure in situ by evaluating laser speckle and optical scattering of FSF data.

Fig. 1

The transmission electron microscope (TEM) image of reconstituted high-density lipoprotein (rHDL) and superparamagnetic iron-oxide nanoparticles (SPION). The rHDL particles, composed of a $1:125$ molar ratio of ApoA-I to phospholipid, appear light, against a background staining of phosphotungstic acid, the SPION are black. The field of view is $250\times 250\mathrm{nm}$, the diameter of the iron oxide particles is 12 nm. The smallest rHDL particles are disk shaped and have approximate dimensions of 3 by 10 nm. The upper inset in each panel is the noise function $B[q(x,y)]$ of that image, the lower inset is the image with the noise signal subtracted $T[q(x,y)]+S[q(x,y)]$, as described in the text. (a) Viscous fluid. Between 5 and 15 particles stack together to form rouleaux of $\sim 30\times 50\mathrm{nm}$. (b) Jammed solid. Some phase transition is evident, as the individual rHDL disks begin to fuse into longer structures, and create complicated interactions with the SPION. (c) Phase transition. Continuous membranes form and the SPION are trapped within the fused rHDL structure.

2.

Method

The equation for Fourier spectrum for spatial frequencies can be simplified as¹⁶

But the fact that the function $A[q(x,y)]$ is image-specific means that although we can uniquely identify individual images in this way, unless the two images are very close to identical, correlation of symmetry between images will be qualitative at best. This will not provide a sufficient basis upon which to distinguish one fluid microstructure state from another, since the actual information acquired from an imaging plane will be unique for each target preparation. In the case of DDM, or LSI, integrating over the entire region of interest provides a spatial averaging that smoothes out image-specific details that differ between different images of the same specimen.² The disadvantage of this method is that it reduces all spatial information in the image to a single scalar value. Although use of the power spectrum in the FSF analysis restores much of the information lost by integrating, it is still an incomplete descriptor of the original object, as phase information is discarded.¹⁷ However, in the instant application, the observation can be made that the characteristic frequencies we seek must exist independently of our ability to reconstruct a visual object from them. To extract a true characteristic spectrum, we first note that the power spectrum of the FT $F{[q(x,y)]}^2$ for any waveform $q(x,y)$, is a vector-valued function in frequency space where each frequency is a linearly independent basis vector. As a sum of linearly independent vector-valued functions, the power spectrum can be decomposed into the following component spectra: (1) a “true”, idealized periodic function, $T[q(x,y)]$, being a family of well-defined characteristic spatial frequencies which we hope to find, (2) a specific signal $S[q(x,y)]$, that completes the image, and (3) noise from instrumentation and detection, $B[q(x,y)]$, which we wish to eliminate. This allows us to rewrite Eq. (3) as

By definition, the characteristic frequencies, $T[q(x,y)]$, will not be found in the image-specific function $S[q(x,y)]$, or vice versa, which means that $T[q(x,y)]$ and $S[q(x,y)]$ are orthogonal, by the standard definition of orthogonality in vector analysis: their inner product is zero. This definition gives us, when applied to Eq. (4):

As can be seen from Eq. (5), even if we remove image-specific frequencies by taking the inner product with a known $T[q(x,y)]$, noise will always be proportional to signal, a factor of sensitivity and resolution that is common to all Fourier spatial analysis applications. Even worse, signal variance is heteroscedactic; doubling the sample area doubles the sampling rate, exponentially increasing the high-frequency noise. The solution is found by a modification of the averaging strategy of LSI and DDM. Rather than averaging over the whole image, we seek a length scale $L$ intermediate to the smallest feature of the image and the limit of detection such that, by averaging the FT of a large number of subregions of this size, a single spatial frequency corresponding to the smallest resolvable object is reinforced, and all other high frequencies are eliminated. Successively averaging larger and larger regions allows us to identifying the key lowest frequency at each length scale. By this step-wise progression, we can eliminate the noise signal. The resulting power spectrum of the noise-corrected FT is the sum of the power spectrum components of the characteristic and specific frequencies, $T[q(x,y)]+S[q(x,y)]$. To determine if two images $A$ and $B$ have similar spatial characteristics, the correlation function $Q(A,B)$ is the inner product of their respective noise-corrected FT:

3.

Materials

Specimens were prepared consisting of phospholipid-coated SPION, and rHDL particles, by a modification of the method established in Ref. 18. Briefly, 1-myristoyl-2-hydroxy-sn-glycero-3-phosphocholine (MHPC, Avanti Polar Lipids, Alabaster, Alabama, cat. no. 855575) was dissolved in a $1:4$ mixture of methanol:chloroform, and deposited on a glass vial as a thin film by solvent evaporation, subsequently hydrated with deionized water and sonicated to form a cloudy suspension. The lipid suspension was heated to 70°C and a suspension of 10 nm d. oleic acid coated ${\mathrm{Fe}}_2{\mathrm{O}}_3$ nanocrystals (Ocean Nanotech, Springdale, Arkansas cat. no. SOR-10-50) in chloroform was added dropwise to the stirred mixture to remove the solvent. Apolipoprotein A-I (Sigma, St. Louis, Missouri cat. no. A0722) at a molar ratio of $1:125$ protein:lipid was added, and the mixture was allowed to cool at room temperature overnight. Successful formation of nanometer-sized particles was confirmed by the clear appearance of the solution, and size measurement by dynamic light scattering.

Images were acquired by negative-stain TEM as previously reported.¹⁹ The sample was transferred to an ammonium acetate buffer (described in Ref. 20), then 10 μl was diluted $1:1$ with 2% phosphotungstic acid. This solution was then drop-cast onto a carbon-coated Formvar-covered copper grid. Samples were allowed to dry and imaged using a Hitachi 7650 TEM coupled to a Scientific Instruments and Applications, (Duluth, Georgia) digital camera and operating at 80 kV.

Sample dispersions are heterogeneous, but they are isotropic with respect to the image frame, and the FT of the region is therefore rotationally invariant. This allows us to extract a one-dimensional trace at the dominant frequency as a representative spectrum for the image. Spatial averaging was accomplished by dividing each image into $N$ subregions, as described in the Appendix, taking the FFT of each region, and calculating a pixel-by-pixel mean for the ensemble of images, $Q_{\mathrm{avg}}(x,y)=1/N\mathrm{\Sigma }{q}_i(x,y)$, where $i$ ranges from 1 to $N$.

All numerical operations were written in elementary Matlab^® (MathWorks, Natick, Massachusetts) functions, the 2-D FFT was calculated using the Matlab built-in function FFT2.

4.

Results

Assembled rHDL particles are on the order of 10 nm, but can stack together to form a secondary structure called rouleaux as shown in Fig. 1(a), with a length scale of 10 to 100 nm, or fuse with each other to form a lipid bilayer membrane with a thickness on the order of 10 nm, but a lateral ordering on a scale $>1000\mathrm{nm}$, as shown in Fig. 1(c), or a state intermediate to the two, shown in Fig. 1(b).

To check if the stepwise spatial averaging method was removing characteristic information as well as background noise, we reconstructed background images from the averaged FT by inverse the FT, which is shown as an inset in Fig. 1(a). There is a qualitative similarity to the background of all images, which is consistent with the fact that they come from similar preparations and instrumentation. By inspection, we can determine that recognizable features from the images are almost entirely absent, giving visual corroboration that the averaging method is not removing characteristic signals.

To determine if our method could recover a known spatial frequency from dissimilar but related model images, we prepared an image of a noise-obscured sine-wave pattern, and a second model image, with a number of alternate sinusoidal frequencies generated by rotation of the original pattern as shown in Fig. 2(a). We then evaluated these images as described above, removing noise through spatial averaging, and generating the power spectrum. As shown in Fig. 2(b), even after removing the noise, the characteristic and image-specific signals do not show an obvious correlation. The identification of a single strong frequency is shown in Fig. 2(c), revealing that the two images do, in fact, share a common frequency.

Fig. 2

Testing on model sine wave image. (a) Comparison was made between two images with the same true spatial frequency, and with unique noise elements present in each image. (b) The Fourier transform of the images in (a) (red trace, open triangles, corresponding to image 1, blue trace, closed circles, corresponding to image 2). (c) The inner product of the family of spatial frequencies identified by the method described in the text identifies the true frequency to be 40 cycles per 1000 pixels.

To demonstrate the elimination of image-specific spatial frequencies between the FT of similar sample preparations, we applied the method to the two images shown in Figs. 3(a) and 3(b). Note that even though image-specific frequencies are not eliminated from their respective power spectra, Figs. 3(c) and 3(d), the inner product of the two reveals striking points of similarity in Fig. 3(e), as predicted by Eq. (7).

Fig. 3

Characteristic spatial frequency correlation of different images of the same sample. (a) and (b) show enlarged regions of images used in the analysis. Each full image was taken under identical conditions, of identical samples, at different locations in the TEM grid. (c) The characteristic and image-specific spatial frequencies $T[q(x,y)]+S[q(x,y)]$ in the text, derived by removing the noise signal, and taking the power spectrum of image (a). (d) The characteristic and image-specific spatial frequencies of image (b). Note the poor correlation between (c) and (d). (e) The inner product of (c) and (d) reveals the characteristic frequency spectrum for the preparation. The spectrum of image (a) is represented as green circles, image (b) as blue triangles, and the inner product as red squares. Of particular note is the frequencies 20 and $32{\mu \text{m}}^{-1}$, corresponding to length scales of the length and width of a single rouleaux (dashed arrows and white boxes). The frequency peak at $40{\mu \text{m}}^{-1}$ corresponds to a length scale of 25 nm, and is most likely the axial view of a single rouleaux. The presence of a frequency was considered significant if it fell within a min-max range of an order of magnitude on the vertical axis, i.e., 0.01 to 0.1 for this figure.

To investigate the utility of this method to differentiate between nanoparticle preparations of dissimilar spatial characteristics, we compared the images shown in Figs. 4(a) and 4(b). The inner product of their respective noise-corrected FT is shown in Fig. 4(c). All samples show a high degree of correlation at high frequencies, and at the frequency corresponding to the spacing between SPION. The absence of spatial frequencies at 20 and $32{\mu \text{m}}^{-1}$ show that there is a significant difference between the two at the length scale corresponding to the dimensions of a rouleaux. To confirm that the spatial frequencies correspond to meaningful physical characteristics, we converted the spatial frequencies to length scales, and identified structures with these lengths that were found in both images. The results of this comparison are shown in insets to Fig. 4(c).

Fig. 4

Characteristic spatial frequency correlation of images of different samples. (a) and (b) show enlarged regions of images used in the analysis. Each full image was taken under identical conditions, of different samples. (c) The inner product of the power spectra of (a) and (b), $T{[q(x,y)]}^2$ in the text. Inset: Features that are found in both images were identified with a size corresponding to the spatial frequency peak indicated by the arrows, from left to right, spacing between SPION $\sim 85\mathrm{nm}$, featureless patch $\sim 25\mathrm{nm}$, repeating region of bilayers $\sim 22\mathrm{nm}$, phospholipid ring around SPION $\sim 18\mathrm{nm}$, and SPION $\sim 12\mathrm{nm}$. Note the absence of signal at $20{\mu \text{m}}^{-1}$ and $32{\mu \text{m}}^{-1}$, which would correspond to the size scale of a rouleaux $\sim 30\times 50\mathrm{nm}$ (dashed arrows and white box).

5.

Discussion

A major obstacle to the use of FSF to characterize imaging planes, in which only intensity information is available, is the difficulty in identifying self-similarity. It is not enough to uniquely identify spatial frequencies within images, but we must also be able to identify similarity within similar images. The preparation of negative stain TEM requires isolation and processing, which makes it impractical to use to identify the presence of different phases of lipoprotein ordering in a tumor, although optical imaging methods, which can be applied in vivo, or directly to excised tissue samples, are not adequate because all the relevant structures are below the diffraction limit of visible light. The use of FSF analysis of laser speckle of optical microscopy has not yet been explored to find colloidal ordering in fluids by our method. A central feature of this type of analysis is to identify the frequencies corresponding to characteristic length scale of patterns within the specimen, as we have shown here. Although it is trivial to distinguish between images that we can see, the characteristic frequency spectrum is crucial for a meaningful spatial analysis of laser speckle from structure that we cannot see. The ongoing goal of this study is to compare the results of the TEM image analysis and laser speckle data from the same samples. Theory predicts that differences in the speckle pattern between samples will be related to structural patterns just as we have shown here for static images.²¹

Recognizing that Newtonian fluid mechanics does not accurately predict nanoparticle transport to cells, current research is focusing increasingly on nanoparticle design in which response to local conditions will change when the agent enters the tumor, either passively, or by chemical, thermal, or photonic activation.²² Experimental strategies have also been proposed to alter intra-tumor pressure by reducing plasma inflow, or increasing lymphatic drainage; to change the effect of filtration or size exclusion by chemically eliminating GAGs, or disrupting collagen structures to increase lateral dispersion of agents.^23,24 These current approaches all recognize that the role of TIF to determine particle distribution is more important than has been previously understood. In light of this, it is vital to develop new methods to characterize the microstructure of the TIF.²⁵ In addition therapies that target barriers to transport formed by shear thickening and jamming will require sensitive tools such as these to evaluate the efficacy of treatment.

6.

Conclusion

The present method provides an elegant solution to the confounding image-specific frequencies found not only in optical applications but for fields as diverse as functional MRI and computer networking.^26,27 By identifying similarities and differences of heterogeneity in these images, we are setting the groundwork to evaluate the microstructure of TIF, and for the evaluation of ECM structure as well. The biological materials used in this analysis are typical in the sense that they form random patterns from a heterogeneous dispersion and show various degrees of similarity at multiple length scales. The method described herein, by providing insight into TIF microstructure, and the distribution of SPION within a principal colloidal component of the TIF, has direct application in quantitative dosimetry of exogenous contrast agents, and can be extended to measure response to therapy. Future work with this method will include an evaluation of laser speckle fields from the HDL preparations as above, and evaluation of nanoparticle distribution patterns in tumor explants.

This research on FSF analysis of TEM images of rHDL and SPION demonstrates the feasibility of recovering characteristic features from an imaging plane of a random heterogeneous pattern without direct reference to feature location. The knowledge that this data can be generalized to a pattern class, and yet is unique within a specific pattern, provides an opportunity to explore the characteristic structures of subdiffraction limit materials, and to better understand their impact on the diagnosis and treatment of cancer.