2.1.

Transmission Dependence of bR on Intensity of Control Beam

We first study the photocontrolled transmission characteristics of bR films, and in subsequent sections we exploit this study for real-time nonlinear optical processing of phantoms and mammograms. Our experiments utilize the dynamics of the $B$ and $M$ states in the bR photocycle shown in Fig. 1 . For all practical purposes we can neglect the remaining short-lived intermediate states of the photocycle. The stable initial $B$ state of a bR molecule absorbs a yellow photon within the broad bR absorption band with maximum at 570 nm, transforms to intermediate states, and then decays to the relatively stable $M$ state within about $50\mu \mathrm{s}$ . The $M$ state can revert to the initial state via a slow thermal process or by a fast photochemical process on excitation with blue light, as the $M$ state has an absorption band at 412 nm. The photochemical stability of the films and high forward- and reverse-quantum yields²⁰ make it possible to use relatively low powers of milliwatts to switch between $B\leftrightarrows M$ states. The lifetime of the $M$ state depends on the reprotonation process and can be varied from a few milliseconds to tens of seconds or even longer using chemical or bioengineering techniques.

Fig. 1

Photoinduced cycle of bR with the two intermediate states of importance for the study. The $B$ state absorbs a yellow photon within the broad bR absorption band with a maximum at 570 nm. This transforms into intermediate states (not shown) and then decays to the relatively stable $M$ state within $50\mu \mathrm{s}$ . The $M$ state is thermally transformed into the initial $B$ state; the lifetime of this transition depends on the pH of the bR film. The $M$ state can also be converted into the bR state within 200 ns by absorption of a blue photon.

Thoma et al. theoretically studied light controlled modulation of the transmittance of a bR film at 568 nm exposed to red or blue light at the same time.¹³ They also studied transmission of a 413 nm beam when simultaneously exposed to yellow light. The absorption $\alpha ({\lambda }_i,I_j)$ of a light of wavelength ${\lambda }_i$ with intensity $I_j$ is given by Eq. 1 as superimposition of the absorption coefficients ${\epsilon }_B\left({\lambda }_i\right),{\epsilon }_M\left({\lambda }_i\right)$ , and the populations of $B\left(I_j\right)$ and $M\left(I_j\right)$ states,

We performed a simple experiment, with the arrangement shown in Fig. 2 , to demonstrate intensity-dependent transmission characteristics of a bR film at 442 nm with a 568 nm control beam and vice versa. He-Cd laser is used for a 442 nm source, while an Ar-Kr laser is used for 568 nm. Neutral density filters ( ${\mathrm{NDF}}_1$ and ${\mathrm{NDF}}_2$ ) are placed in the path of both the laser beams to control the intensity. The two laser beams are spatially overlapped on the bR film. The wild-type bR film is purchased from Munich Innovative Biomaterials GmbH (Munich, Germany). It has optical density of about 5 at 570 nm with a thickness of $\sim 100\mu \mathrm{m}$ . The required power measurements to demonstrate the intensity-dependent transmission characteristics were carried out using Newport power∕energy meter model 1825C. The results displayed in Fig. 3 show the transmission of bR film at 442 nm decreases with an increase in 568 nm control beam intensity. Similarly, the results displayed in Fig. 4 show that the transmission of the film at 568 nm probe beam decreases with increase in 442 nm control beam intensity. The experimental results shown in Figs. 3, 4 are in agreement with the simulated results of intensity-dependent transmission of bR films given by Thoma ¹³ In the bR photocycle, the molecules in the initial $B$ state absorb 568 nm light and undergo photoisomerization to $M$ state, which has the absorption band in the blue region. The $M$ state molecules can isomerize back to the initial $B$ state on excitation with blue light. The experimental results indicate that the relative population of the two states can be controlled by simultaneous illumination of the film with 442 nm and 568 nm beams. The ratio of the $B$ and $M$ state population depends on the relative intensities of the two beams. With increase in the 568 nm intensity, more $M$ state molecules are produced, shifting the absorption to the blue region. On the other hand, with increase in intensity of the 442 nm beam, while keeping the 568 nm light at constant intensity, the molecules in the $B$ state dominate with the absorption band shifting to the yellow region.

Fig. 2

Experimental setup used to study the transmission dependence of bR on control beam intensity. NBF: narrow band filter to block the stray yellow light; ${\mathrm{NDF}}_1$ and ${\mathrm{NDF}}_2$ are neutral density filters to control the intensity of the beams.

Fig. 3

Transmission of bR films at 442 nm probe as a function of intensity of 568 nm control beam.

Fig. 4

Transmission of bR films at 568 nm probe as a function of intensity of 442 nm control beam.

Our results shown in Figs. 3, 4 demonstrate an important feature—that the probe transmission of one wavelength can be significantly reduced by the control beam of a different wavelength in the high intensity region. Based on these results, we designed a scheme of spatial filtering for image processing. For example, if the information-bearing blue beam (442 nm) is Fourier transformed to the bR plane, the desired spatial frequencies can be selectively blocked by illuminating the film with a 568 nm control beam. Since different spatial frequency components are spatially separated in the Fourier plane, the location of the control beam on the bR film determines the components blocked. If we desire to block only low frequency components, the control beam can be simply focused to the center of the Fourier spectrum.

2.2.

Processing of Phantom Objects and Clinical Film Mammograms

Figure 5 shows the experimental setup for edge enhancement of phantoms and mammograms based on this mechanism of photoisomerization process of bR molecules using two wavelengths, 442 and 568 nm. A spatially filtered and well-collimated 442 nm beam from a He-Cd laser illuminates the object, either a phantom or a clinical mammogram. A 12.5-cm focal length lens $\left(L_1\right)$ is used to obtain the Fourier spectrum of the object. The bR film is placed at the Fourier plane of the blue beam. The inverse Fourier transformation is obtained using 10.5-cm focal length lens $\left(L_2\right)$ and the object is imaged onto the CCD detector placed at the back focal plane of the second lens.

Fig. 5

Experimental setup for processing film mammograms using nonlinear optical filtering in bR films with two wavelengths; $L_1,L_2$ , and $L_3$ are converging lens, $f_B$ and $f_Y$ are focal lengths of $L_1$ and $L_2$ , respectively, and NBF is a narrow band filter to block 568 nm at the CCD plane. The spatial overlap of two beams at the bR film plane (which is normal to the optic axis of the blue laser beam) is shown in the plane of the paper in the inlet. ${\mathrm{CL}}_1$ and ${\mathrm{CL}}_2$ are the collimation lenses. ${\mathrm{NDF}}_1$ and ${\mathrm{NDF}}_2$ are neutral density filters to control the intensity of the beams.

A second collimated 568 nm control beam from an Ar-Kr laser is tightly focused on to the bR film using converging lens $\left(L_3\right)$ of focal length 10.5 cm. For blocking low spatial frequency components of the object, the zero orders of both the wavelengths (focal spots sizes) are spatially overlapped on the film, as shown in the inlet of Fig. 5. This is achieved by proper scaling of focal spot sizes of lenses $L_1$ and $L_2$ such that their focal lengths $f_B$ and $f_Y$ satisfy the following relation:

The protein bR, an optically switchable bistable material, is used to process the Fourier spectrum of the mammogram containing information about the microcalcifications (high spatial frequencies) and background of soft tissue (low spatial frequencies). In the absence of any control yellow beam, the entire information carried by the blue light is passed through bR without any processing, as blue photons are not absorbed by the bR, which is in its stable state $B$ . In actual experiments, lens $L_3$ focuses the control beam such that its focal spot size precisely overlaps the focal center spot size of blue light. This leads to the local reduction of transmission of low spatial frequencies in the blue pattern, as the bR molecules absorb the yellow control beam and undergo photoisomerization from $B$ to $M$ state, for which the absorption peak is located in the blue region. Hence the transmitted pattern in the blue light captured by the CCD is predominantly of high frequencies corresponding to microcalcifications. Moreover, an edge enhanced version of the mammogram is observed on the TV monitor. When the object mammogram is changed, the center of the low spatial frequencies remains at the same focal spot, but their spatial extent changes. In conventional optical filtering techniques, one has to first properly scale the size of the high pass filter and then position it properly at the Fourier plane. But the scheme proposed here has the advantage of not requiring the tight specifications of size and location of the spatial filter at the Fourier plane. One can change the filter size with ease just by moving the lens or by controlling the yellow light intensity using a neutral density filter.

As mammograms consist of wide spectra of breast lesions that have various structural distortions, the feasibility of the technique is established initially with different shapes and sizes of phantom objects, as shown in Figs. 6(a), 6(c), 6(e) . Different shapes of objects give rise to different types of spatial frequency spectra at the Fourier plane where bR is placed, and it is important to demonstrate that the technique works for processing any complex frequency spectrum. For example, the image in Fig. 6(a), the letter E, consists of only horizontal and vertical edges and therefore its high spatial frequency spectrum at the Fourier plane lies only in vertical and horizontal directions, respectively. However, the image in Fig. 6(b) is circular in shape and has circular edges, therefore its high spatial frequency components at the Fourier plane are also distributed in a circular manner without any preferential direction. The image in Fig. 6(e) has an arc-shaped edge in addition to horizontal and vertical edges, therefore its high spatial frequency components at the Fourier plane are distributed in circular, vertical, and horizontal directions. The clinical mammogram is a complex image and consists of many such circular, horizontal, vertical, or combination edges related to microcalcifications and other architectural distortions in the human breast, and their spatial frequency distribution at the Fourier plane is complex.

Figure 6 shows the results of phantom objects processed using the technique. Unprocessed images in Figs. 6(a), 6(c), 6(e) are captured by the CCD camera. The phantom objects used in the experiment include the transparent letter E with little artificial distortions (pointed by the arrows), a transparency with concentric white and black circles, and an arrow. Besides the edges of letter E, these narrow artificial distortions also contribute to the high spatial frequencies in the Fourier spectrum of the image. Now in the presence of tight focusing of the control beam, the yellow light locally weakens the low spatial frequencies carried by the blue beam, and allows only high spatial frequencies to pass through bR. Thus the reconstructed image captured by the CCD camera, as shown in Fig. 6(b), is edge enhanced, clearly showing the artificial distortions in addition to the edges of the E. Similarly, the reconstructed image for concentric bright circles is also edge enhanced, as shown in Fig. 6(d), clearly showing the bright edges at the crossing between the dark and bright concentric circles. The edge enhanced image of an arrow in Fig. 6(e) is shown in Fig. 6(f). The results demonstrate the enhancement of high spatial frequency components for different shapes and sizes of object, establishing the feasibility of the technique for enhancement of microcalcifications and other abnormalities in mammograms.

Fig. 6

(a), (c), and (e) represent original images of E, concentric bright circles and an arrow, respectively, captured by CCD camera in the absence of yellow light using the experimental setup shown in Fig. 5. (b), (d), and (f) represent respective processed images captured by CCD camera in the presence of yellow light.

Now we study analog screen film mammograms. Figures 7(a), 7(d) display scanned version of original film mammograms used in the experiment. Figures 7(b), 7(e) display the respective scanned versions of ROIs. The ROIs in the original film mammogram are illuminated by blue light, as shown in the experimental setup in Fig. 5. As discussed previously, the yellow control beam induces photoisomerization in bR film, thus reducing the transmission of low spatial frequency components. The background of soft dense breast tissue corresponding to low spatial frequencies in the Fourier spectrum is filtered out at the bR plane. On the other hand, the high spatial frequencies corresponding to microcalcifications are transmitted. Figures 7(c), 7(f) show the reconstructed image displaying only microcalcifications not visible to the naked eye in the original clinical mammogram on the TV monitor connected to the CCD.

Fig. 7

(a) and (d) represent a scanned version of film mammogram used in the experiment with ROI encircled; (b) and (e) are respective magnified ROI; (c) and (f) are the corresponding processed image of ROI in a film mammogram.

2.3.

Processing of Digital Phantoms and Digital Mammograms

Digital mammography is becoming popular for early detection of breast cancer, as it incorporates modern electronic and computer techniques. Once the images have been obtained by the digital machines, they can be electronically manipulated—the physician can zoom in, magnify, optimize contrast, and conveniently look at different regions of breast tissue on the monitor. Though digital mammography offers distinct advantages over conventional film, it also presents new challenges to the user in terms of image acquisition, storage, archiving, interpreting, and processing huge amounts of information.^{21, 22, 23} The nonlinear optical Fourier filtering technique proposed in this study for clinical mammograms is also useful for processing digital mammograms. An electrically addressed spatial light modulator (SLM) is incorporated into the experiment shown in Fig. 5 to facilitate the interface between a digitally stored mammogram in the computer and the optics used in the experiment. With state of art SLMs like Boulder Nonlinear Systems, a $512\times 512$ multilevel∕analog liquid crystal spatial light modulator, commercially available with many features such as temporary image storage, programmed extraction, and enhancement of ROI, the technique can be adopted as hardware in digital machines for enhancement of microcalcifications in digital mammograms.

The experimental setup shown in Fig. 8 is used to process digital phantoms and clinical mammograms; either one, stored in the computer, is displayed onto the electrically addressed SLM (EASLM, Kopin 320 Mono cyber display) using the SLM driver. The collimated He-Cd 442 nm laser beam illuminates the SLM, and the output of the SLM is a coherent optical signal bearing the image displayed on the SLM. The resulting optical image is then processed using the nonlinear optical filtering technique as discussed in Sec. 2.2 for film mammograms.

Fig. 8

Experimental setup for processing digital mammograms and phantoms using nonlinear optical filtering in bR films with two wavelengths; $L_1,L_2$ , and $L_3$ are converging lenses, $f_B$ and $f_Y$ are focal lengths of $L_1$ and $L_2$ , respectively, and NBF is a narrow band filter to block 568 nm at CCD plane. ${\mathrm{CL}}_1$ and ${\mathrm{CL}}_2$ are the collimation lenses. ${\mathrm{NDF}}_1$ and ${\mathrm{NDF}}_2$ are neutral density filters to control the intensity of the beams.

To demonstrate the feasibility of the technique for real clinical digital mammograms, initially we studied some digital phantoms shown in Figs. 9(a), 9(c) . Figure 9(a) shows the letter E, which consists of horizontal and vertical edges. This image is displayed onto the SLM from the computer. The output of the SLM, the coherent optical signal bearing the image of letter E, is now Fourier transformed using the biconvex lens $L_1$ . The spatial frequency spectrum of this image is obtained at the back focal plane where bR film is placed. The lens $L_3$ tightly focuses the yellow light such that the zero orders of both the wavelengths (focal spots sizes) are spatially overlapped on the bR film. This locally filters out the low spatial frequencies in the blue beam and allows only high spatial frequencies through the bR film. Thus the reconstructed image captured by the CCD camera, as shown in Fig. 9(b), is edge enhanced. Figure 9(c) shows a digital phantom containing white simulated microcalcifications buried in gray background. Contrast between white pixels used to simulate microcalcifications and surrounding gray pixels used to simulate background of soft dense breast tissue is so small that the microcalcifications are not visible to the naked eye. In the presence of yellow light, low spatial frequencies (gray background) are blocked, and Fig. 9(d) shows the reconstructed image with only high spatial frequencies corresponding to the simulated microcalcifications now visible to the naked eye.

Fig. 9

(a) and (c) represent original images of E and phantom with simulated microcalcifications with white spots buried in the white background with little gray level difference; (b) and (d) represent corresponding processed images reconstructed with high spatial frequencies.

Figure 10 shows results for a clinical digital mammogram. Figure 10(a) is the original digital mammogram (with ROI encircled with black ink) displayed on the SLM from the computer monitor. The digital information is converted into coherent optical information and then processed as described earlier. Figure 10(b) shows the digitally magnified ROI and Fig. 10(c) shows the reconstructed image with high spatial frequencies captured by CCD clearly showing the microcalcifications with good contrast.

Fig. 10

(a) Digital film mammogram used in the experiment with ROI encircled; (b) magnified ROI; and (c) processed image of ROI of (a).