1.

Introduction

Spectral object recognition and detection are based on a group of image processing techniques which are referred to as hyperspectral imaging (HSI).^1,2 Acousto-optic tunable filters (AOTFs) are a kind of adaptive dispersive device commonly used for HSI.^3–12 Transmitted wavelength at the output of an AOTF depends on the frequency of the controlling RF signal. Purely electronic tuning with a switching time on the order of $10\mu \mathrm{s}$ and the absence of moving parts makes AOTFs suitable for real-time operation. A typical numerical aperture of imaging AOTFs approximately equals to $F/10$, which makes them easy to use with light microscopes in wide-field applications. Several HSI systems based on AOTFs are commercially available now. Spectral image processing by means of AOTFs can also be used in various image processing techniques, for instance, optical coherence microscopy¹³ and imaging spectropolarimetry.^14–17

In the case of monochromatic coherent input light, AOTFs perform spatial filtering of optical beams and can be used for such image processing operations as optical phase detection, edge enhancement, and low-pass and high-pass spatial filtering.^18–25 Optical phase detection by means of AOTFs is based on the angular selectivity of acousto-optic Bragg diffraction.^19,24 Diffraction efficiency varies for different fragments of the curved wavefront after the phase object, resulting in amplitude modulation of diffracted light. Low-pass and bandpass spatial filtering regimes are based on dependency of the transfer function of the AOTF on the frequency of the controlling RF signal.^18,20–24

HSI techniques have recently gained recognition in cancer diagnostics.^11,26–38 HSI can be combined with different object illumination modes: reflection, transmission, or fluorescence. Spectral characteristics of tissues can be used for differential diagnostics of benign and malignant tumors and for detection of cancer cells in cytological smears and histological sections. Since AOTFs are adaptive in both the spectral and spatial domains, a combination of these features can be used for improvement of HSI methods. Complementary use of HSI and contrast enhancement by means of the same AOTF-based image processing system can be helpful for better visualization of details of translucent objects.

Contrast enhancement is a relevant problem in microscopy when translucent objects are under examination. Such objects are live cells and tissues, unstained cytological smears after fine needle aspiration biopsy, and histological sections. Phase-contrast wide-field imaging is a classical technique for visualization of phase-only objects. An alternative approach to this problem is based on interferometric techniques using coherent illumination of samples. These are generally referred to as quantitative phase imaging (QPI).^39,40 Most QPI methods rely on computational phase retrieval algorithms from detected holograms or interferograms, which makes them highly informative but relatively slow. Some real-time QPI methods have been developed using fast computational algorithms.^41–43 Unlike them, acousto-optic image processing techniques directly affect the angular spectrum, resulting in intensity modulation, and do not require computational phase retrieval algorithms. Thus acousto-optic methods of contrast enhancement can be considered as functional extensions of HSI which can be used both in research and in clinical practice. AOTF-based image processing systems can be used with standard bright-field microscopes.

Morphological diagnosis of malignancy is based on a method of light microscopy of cytological smears or histological sections of surgically removed tissues. Sometimes a problem of express examination of frozen histological sections without staining procedures could arise at the time of surgery. A problem of morphological determination of human tumor malignancy is sometimes a crucial one. For example, 10% of surgically removed follicular thyroid tumors are classified by surgical pathologists as “well-differentiated tumors of uncertain malignant potential.”⁴⁴ Doubtful features of capsular tumor invasion and focal morphological nuclear changes of tumor cells are the reason to distinguish a group of borderline thyroid tumors that are not widely recognized as a separate World Health Organization classification group. This means that the malignancy of this tumor group needs to be more precisely determined. It was previously suggested by Kakudo et al.⁴⁵ and by Abrosimov et al.⁴⁶ that some benign and borderline thyroid tumors are malignant neoplasms with low malignancy grade and favorite prognosis. The suggestion of low grade of malignancy is made on the basis of literature review and analysis of the results of immunomorphological and molecular studies. It could be expected that some modern image processing techniques would help to distinguish benign and malignant thyroid tumors and develop additional morphological criteria for differential diagnosis. That is why we made the comparative study of plain bright-field microscopy and acousto-optically enhanced morphological features of unstained samples of malignant tumors (papillary thyroid carcinoma).

In this paper, we developed an instrumental method for contrast enhancement in microscope slides using filtration of angular spectral components in an AOTF assisted with a telecentric lens system. The method was commissioned during analysis of histological sections of human papillary thyroid carcinoma. The results were compared with HSI of the samples performed by the same AOTF using white-light illumination in transmission mode.

2.

Image Processing Method

For analysis of HSI and edge enhancement using AOTFs, we focused on peculiarities of Bragg phase matching during anisotropic diffraction in uniaxial crystals. Noncritical phase matching (NPM) geometry of acousto-optic interaction take place when the tangents to the wave normal surfaces for incident and diffracted light are parallel to each other, resulting in a wide acceptance angle of the filter. Thus, high spectral and spatial resolution is obtained.³ Noncollinear acousto-optic interaction is characterized by the circular type of the transfer function in the NPM configuration.^23,24

Noncollinear AOTFs are based on uniaxial birefringent crystals.³ The diffraction plane is perpendicular to the plane of the piezotransducer (PZT) and contains the optical axis of the crystal. The input light beam is considered as a composition of plane-wave components of the angular spectrum. In the following analysis, the direction of each of the input light plane waves is determined by two angles: $\varphi$ is the angle between the wave vector and the plane of diffraction, and $\theta$ is the Bragg angle (i.e., the angle between the plane of the PZT and the wave vector of light). The frequency of ultrasound $f$ that provides exact phase matching at the wavelength $\lambda$ can be expressed as a function of the angles $\theta$ and $\varphi$ as follows:

Calculations of the phase-matching frequency are shown in Fig. 1 for a paratellurite noncollinear AOTF. The center of the analyzed region was chosen as $\theta =15\mathrm{deg}$ which approximately corresponds to the NPM Bragg angle for the PZT tilt angle of 11 deg relative to the [001] axis in the crystal. Those parameters correspond to the AOTF described in Sec. 3 and used in experiments. The equal frequency contours are closed around the NPM point. Those contours can be treated as parametric curves coupling $\theta$ and $\varphi$ which provide maximum diffraction efficiency at the given values of the wavelength $\lambda$ and the ultrasonic frequency $f$. The transfer function of the AOTF is localized in an area around the phase-matching contour. Thus, by varying the frequency of ultrasound which is applied to the PZT of the AOTF, one can transmit higher or lower spatial frequencies in the angular spectrum of light.

Fig. 1

Color-mapped plot of Bragg phase matching frequency $f$ versus the direction of plane wave propagation in the paratellurite-based AOTF. PZT is tilted to the angle 11 deg relative to the [0 0 1] axis of the crystal; optical wavelength $\lambda =633\mathrm{nm}$.

Visualization of the AOTF transfer function in Fig. 2 was performed by illuminating the filter with a focused laser beam. Zero spatial frequency of the beam corresponds exactly to the NPM point, $\theta =14.9\mathrm{deg}$ and $\varphi =0$. The scale along the $\theta$ (horizontal) and $\varphi$ (vertical) axes in the experimental plots corresponds to that in simulations. When the AOTF is tuned exactly at NPM, Figs. 2(a) and 2(d), low-pass spatial filtering takes place. A slight increase in the driving frequency, as in Figs. 2(b) and 2(e), results in a wider range of spatial frequencies transmitted with efficiency 50% and higher. Further increase in the frequency $f$ results in a ring type of the transfer function, as shown in Figs. 2(c) and 2(f). The transfer functions can be observed only with monochromatic input light. In the HSI operation mode of the AOTF, when the input light is broadband, phase matching for higher spatial frequencies takes place at a lower wavelength than for lower spatial frequencies.

Fig. 2

(a, b, c) Calculation and (d, e, f) experimental measurement of the AOTF first-order transfer function for coherent light at $\lambda =633\mathrm{nm}$: (a, d) exact Bragg phase matching at the center, $f=132.8\mathrm{MHz}$; (b, e) maximum FWHM angular aperture, $f=132.9\mathrm{MHz}$; (c, f) bandpass filtering, $f=133.0\mathrm{MHz}$. Transmitted intensity is encoded as linear gray scale. Dashed lines indicate the angular scale in terms of the beam focal ratio. $\theta$ corresponds to the $x$ axis; $\varphi$ corresponds to the $y$ axis.

3.

System Design

The experimental setup is shown in Fig. 3. The sample is illuminated with an He–Ne laser, $\lambda =633\mathrm{nm}$, through the fiber collimator, single-mode fiber patch cord, and a bare fiber terminator located in the front focal plane of the collimating lens. A ground glass diffuser is used to reduce coherent artifacts and improve image quality. Thus, almost plane wave illumination of the object is obtained. The transmitted light wave is modulated by the object both in phase and amplitude. A primary image of the sample is formed at the side port of the inverted microscope (Nikon, Ti-E) by means of the microscope objective (MO; Nikon, Plan Apo $\lambda$, $20\times /{0.75}^{\prime \prime }$) and the tube lens. The field lens is located near the primary image plane to provide proper beam path and eliminate vignetting by the elements of the subsequent image relay system. The telecentric diaphragm (TD) is located in the front focal plane of the lens (${\mathrm{RL}}_1$) which relays the image to the output plane of the AOTF. Crossed polarizers (${\mathrm{P}}_1$ and ${\mathrm{P}}_2$) provide the vertical polarization at the input of the AOTF and transmit diffracted horizontal polarization at the output of the AOTF. The diffraction plane is parallel to the breadboard. The second relay lens (${\mathrm{RL}}_2$) images the diffracted field at the CCD matrix (Apogee, Alta U32, $6.8\mu \mathrm{m}/\text{pixel}$). The output diaphragm (OD) is used to stop the residual stray light from the zero diffraction order. Monitoring of the sample is performed with a color CCD (Nikon, DS-Fi2, $3.4\mu \mathrm{m}/\text{pixel}$) placed at the eyepiece tube side port of the microscope.

Fig. 3

(a) Schematic and (b) photograph of the experimental setup: AMP, RF power amplifier; CCD, CCD camera; CL, illumination collimator lens; DG, diffuser glass; FC, fiber collimator; FL, field lens; FT, fiber terminator; GEN, RF signal generator; RL, image relay lenses; MO, microscope objective; OD, output diaphragm; P, polarizers; S, sample; SMF, single-mode fiber patch cord; SP, side port; and TD, telecentric diaphragm.

Telecentric optical systems have proved to be efficient in compensation for chromatic aberrations in AOTF imaging spectrometers.^6,8 In addition to that, they provide another advantage that is utilized in our experimental setup: the TD precisely controls the angular spectrum of the light beam incident at the AOTF. Opening of the diaphragm limits the maximum spatial frequency at the input of the AOTF. As a result, the spatial spectrum of the light wave from the object is matched with the acceptance angle of the AOTF.

The imaging AOTF is based on the paratellurite single crystal with clear aperture of 15 mm and angular acceptance of 6 deg that was specially designed and fabricated in-house for this research. The passband of the AOTF was equal to 0.8 nm at the wavelength 633 nm, which corresponds to a spectral resolution $20{\mathrm{cm}}^{-1}$. Refraction of light at the tilted output facet of the AOTF takes place in such a manner that the diffracted beam is parallel to the input beam, while the zero-order beam is deflected and blocked by the OD. This configuration of the AOTF ensures zero astigmatism of the optical system and facilitates its precise adjustment, since all optical elements from the telecentric diaphragm to the CCD are coaxial. For the purpose of making the optical setup more compact, we additionally used a periscope to wrap the beam.

For comparison of the contrast enhancement performance with conventional bright-field microscopy and HSI, it was possible to use incoherent white-light illumination without changes in the optical setup at the detector end. For that purpose, the laser illuminating system was disabled and a conventional microscope condenser was used for illumination of the sample. The frequency of the ultrasonic wave $f$ remained unchanged; hence the transmitted by the AOTF wavelength $\lambda$ was equal to 633 nm, that is, the same as the laser emission.

4.

Results and Discussion

The experimental system for image processing was commissioned using the samples of histological sections of human thyroid tumors. The results in Fig. 4 demonstrate two serial histological sections of a follicular variant of papillary thyroid carcinoma obtained with an objective with magnification of $20\times$. Fragments (a) to (c) were captured using the first sample stained with hematoxylin and eosin (H&E); fragments (d) to (i) were captured for the identical second sample that was unstained. The original spatial resolution of the fragments was $1920\times 1920\text{pixels}$ for color images in Figs. 4(a) and 4(d), $480\times 480\text{pixels}$ for the fluorescence image in Fig. 4(f), and $960\times 960\text{pixels}$ for the rest of the images processed with the AOTF.

Fig. 4

Fragment of histological section of papillary thyroid carcinoma: (a) bright-field color image of the H&E-stained sample; (b) bright-field spectral image of the H&E-stained sample; (c) image of the H&E-stained sample with coherent illumination and matched spectral filtering; (d) bright-field color image of the unstained sample; (e) bright-field spectral image of the unstained sample; (f) autofluorescence image of the unstained sample; (g) image of the unstained sample with coherent illumination and matched spectral filtering; (h) image of the unstained sample with coherent illumination and inverted contrast; and (i) image of the unstained processed in bandpass filtering mode.

Noise in raw signal images originated from constant spurious flare of the dark current of the CCD at room temperature ($+20°\mathrm{C}$). The background flare signal of the CCD was captured with the AOTF switched off and further subtracted from raw signals, resulting in analyzed images. The residual noise in the analyzed images is characterized with the standard deviation of the CCD dark current. Spectral images in Figs. 4(b) and 4(e) were captured with an exposure time of 0.5 s. Images with laser illumination and AOTF spatial filtering were captured with an exposure time of 5.0 s. Global mean values and standard deviations of absolute pixel counts are summarized in Table 1 for unprocessed signal images and for corresponding spurious background images. The mean value of the dark current was equally contributing into signal and background images, and therefore it was canceled after subtraction of the background flare. Calculations of noise level for color images shown in Figs. 4(a) and 4(c) were not available because dark current measurement could not be correctly performed by the CCD software. Analysis also was not performed for the fluorescence image in Fig. 4(f) because that image was captured only for the purpose of identification of the unstained sample with the stained one.

Table 1

Comparison of noise level and contrast obtained in different image processing methods.

In the conventional bright-field color image of the stained sample, Fig. 4(a), both intracellular components, such as nuclei and cytoplasm, and intercellular structures of stroma and colloid are clearly distinguishable. A spectral image of the same fragment, Fig. 4(b) was obtained using bright-field white illumination of the sample and spectral filtering by means of the AOTF of the experimental setup. The lower spatial resolution of the spectral image is a result of aperture limitation in the relay optical system. Using laser light for illumination of the sample in Fig. 4(c) and filtering with the AOTF at the phase-matching frequency $f=132.80\mathrm{MHz}$ resulted in a better contrast and higher depth of field than in white-light spectral imaging mode. The unstained histological section in Fig. 4(d) demonstrates the same fragment of the tumor, but the tissue structures cannot be visualized. The same level of detail with a slightly higher contrast is obtained using bright-field white illumination with spectral imaging; the results are shown in Fig. 4(e). One can only see the contours of tissue structures. For the purpose of confirmation of the identity of serial samples, the unstained one was additionally imaged in autofluorescence mode with laser excitation at the wavelength of 532 nm. The flat spot of laser emission was illuminating the central part of the field of view by means of the epifluorescence optical port of the microscope. Different intensities of autofluorescence highlight the basal and the apical contours of cell layers. That makes it possible to recognize the same follicular structures as in Figs. 4(a)–4(c). The images of the unstained sample were processed using the acousto-optic method of spatial filtering described in Sec. 2. In Fig. 4(g), the ultrasonic frequency $f=132.80\mathrm{MHz}$ corresponds to NPM exactly at the laser wavelength $\lambda =633\mathrm{nm}$. In the image, the contours of tumor cells and of stroma are seen, and the contrast is higher than in bright-field and spectral imaging modes. The image in Fig. 4(h) was captured at the ultrasonic frequency $f=132.90\mathrm{MHz}$, which caused phase mismatch for the central components of the angular spectrum. As a result, the image has an inverted contrast and better visualization of details. In particular, colloid was made visible. Finally, the image in Fig. 4(i) was captured with extreme phase mismatch at the frequency of ultrasound, $f=133.00\mathrm{MHz}$. In this regime, the AOTF blocks most of the light from the object except for the highest components of the angular spectrum transmitted by the optical system. This operation mode is bandpass filtering in the domain of spatial frequencies, and the maximum of intensity corresponds to the gradients of intensity in the bright-field images. Only the basal contours of layers of the tumor cell are visible. Though the spatial resolution in the bandpass filtering mode degrades, this mode can be used as an additional visualization regime for unstained samples.

Global contrast of the images in Fig. 4 was estimated numerically as the ratio of the root mean square (RMS) of image intensity deviation to the global average intensity of pixels. Color images [Figs. 4(a) and 4(d)] were analyzed for each channel (red, green, and blue) separately. The mean intensity of the three channels with the equal weights of $1/3$ was also analyzed. For analysis of contrast, all the images were converted to normalized pixel values ranging from 0 to 1 and the same size of $960\times 960\text{pixels}$. This image size corresponds to pixel binning $2\times 2$ for the color images and the original size for the rest of the images. The contrast of the fluorescence image was not analyzed because the excitation laser emission was illuminating only the central region of the image, and evaluation of the RMS deviation would not be correct. Numerical data on calculated contrast are given in Table 1.

The bright-field image of the H&E stained sample has different contrasts for different channels with the maximum of almost 30% for the green channel. The spectral image of the same sample has contrast slightly higher than that for the red and blue channels but lower than that for the green channel. The highest contrast of images of the stained sample was obtained using laser illumination and acousto-optic filtering in the low-pass mode. The contrast of the bright-field images of the unstained sample in all channels of the color image is sufficiently lower, that is, about 10%. Nevertheless, the images of the unstained sample obtained with laser illumination and acousto-optical spatial filtering have the highest contrast among all the images. Moderate suppression of central components of the angular spectrum, as shown in Fig. 4(h), does not lead to an increase in image contrast, but it alters visual representation of the sample. Ultimate bandpass filtering, shown in Fig. 4(i), sufficiently alters the image. The contrast of the image is the highest at the cost of the average image intensity. This image processing mode is commonly known as edge enhancement.

The numerical analysis of experimental data demonstrates the improvement of image contrast for unstained samples compared to bright-field and HSI techniques. For practical applications, it is most important to analyze the results of the acousto-optic image processing method for those samples. The effect of contrast enhancement originates from adaptive modification of the transfer function of the optical system. As shown in Sec. 2, the AOTF performs adaptive spatial filtering of coherent light. According to the principles of Fourier optics, field distribution at the image plane is determined by the two-dimensional Fourier transform of the angular spectrum, which is a product of the angular spectrum of the input light and the transfer function of the optical system. Since the Fourier transform is linear, different components of the angular spectrum additively contribute to the image with their amplitudes and phases. Several factors affect the angular spectrum of processed images. First is the numerical aperture of the MO. Second is the cutoff of high spatial frequencies by the telecentric diaphragm. In our experiment, the beams at the input of the AOTF had the focal ratio of $F/16$. Using the AOTF with circular symmetry of the transfer function for spatial filtering, we choose one or another subband of the spatial frequencies to be transmitted. Thus we partially subtract the contribution of either higher or lower spatial frequencies from the image. When we apply the transfer function with partial subtraction of central components, as in Figs. 2(c) and 2(d), we obtain the angular spectrum, which is more uniform than the initial one, and the contrast in small details increases. When the range of transmitted components of the angular spectrum is approaching the cutoff angular frequency, edge enhancement is performed. This image processing mode is shown in Fig. 4(i).

5.

Summary

In this paper, we experimentally demonstrated that using AOTFs for HSI microscopy of biological samples can be complemented with spatial filtering techniques using an additional monochromatic laser illumination unit. Owing to the adaptivity of the spatial transfer function of AOTFs, visualization mode of the samples may be altered from low-pass filtering to high-contrast bandpass filtering. The optical system of the imaging system complies with the output light ports of standard bright-field microscopes and does not require modification when switching between HSI and edge enhancement modes of operation.

The illustrations presented demonstrate that using advanced analog image processing by means of the AOTF-based adaptive image filtering system can improve visualization of certain details in unstained samples of histological sections. Combined with HSI, which is performed by the same optical system, contrast enhancement technique can give more information about the structure and composition of examined objects.