The accurate and complete removal of tumor, with minimal removal of healthy surrounding tissue, is guided by the examination of pathology. The preparation of pathology, either during surgery or after, is time-consuming and labor-intensive. In the setting of nonmelanoma skin cancers, frozen pathology that is prepared during Mohs surgery requires 20 to 45 min per excision, and two or more excisions are performed, such that the total preparation time lasts from two to several hours.1 In other settings such as with head-and-neck and breast cancer, fixed pathology is prepared after surgery. Preparation of fixed sections requires one to two days. Since it takes such a long time to obtain pathology results, the patient is sent home immediately after surgery. If the pathology shows positive tumor margins, the patient must undergo additional surgery (resection), and/or radiotherapy or chemotherapy. The incomplete removal of tumor and positive margins are reported to occur in 20 to 70% of patients, depending on the surgical setting.2,3 Although the rates of resections can be reduced by aggressive removal of tissue from a wide margin around the tumor, it can affect the functionality of the organ or produce unacceptable aesthetical results for the patient. Therefore the objective is to minimize the removal of healthy tissue by reducing the margins and still perform complete removal of the tumor. To address this problem, high-resolution optical imaging methods capable of displaying nuclear and cellular morphology, offer the possibility for rapid detection of tumors in large areas of freshly excised or biopsied tissue. Currently, confocal microscopy, full-field optical coherence tomography, multispectral macro-imaging, multiphoton microscopy, fluorescence lifetime imaging and other approaches are being developed for this purpose.188.8.131.52.184.108.40.206.13.14.15.–16,17
We have been developing confocal mosaicing microscopy as a technology platform for imaging tumor margins in fresh tissue excisions from surgery2,3,1314.–15 to provide high-resolution images of large areas of tissue within minutes. Previously, we collected square-shaped images with aspect ratios of , and stitched them together with custom software into a mosaic to display a large field of view. Mosaicing of images to display of excised tissue from Mohs surgery was demonstrated in 9 min.1314.–15 In a blinded examination of 45 fluorescence mosaics by two Mohs surgeons, basal cell carcinoma margins were detected with an overall sensitivity of 96.6% and a specificity of 89.2%.18,19 The results of this preclinical study demonstrated the promise of confocal mosaicing microscopy.
Although 9 min is certainly faster than the hours or days required for preparing conventional (frozen or fixed) pathology, routine implementation in surgical settings will require shorter processing times. The unequivocal feedback from surgeons was that this technology was unlikely to be adopted unless the speed was improved. The clear expectation was that to reach practical and routine use in any operating room setting, mosaicing must meet the surgeons’ need to examine tumor margins in large areas () within a few minutes. To address this need we designed a faster approach called “strip mosaicing.” The initial instrumentation work for strip mosaicing was reported last year.20 Strip mosaicing is performed with a combination of optical and mechanical scanning. The tissue is mechanically translated across a linearly scanned focused laser beam. This approach is faster because the speed of mosaicing strips is primarily governed by the fast line acquisition rate in the confocal microscope20 and does not use the single frame acquisition method previously reported.14,18
In this paper, we report further advances and enhancement of the electronics and mechanics, the development of custom software, and the overall integration of hardware and software into an approach that demonstrates imaging of of human skin tissue with 1-μm lateral resolution in 90 s and of breast tissue with 0.8-μm lateral resolution in 13 min. The enhancements include a device to mount and flatten fresh tissue from surgery, improvements in the tissue translation stage for speed, accuracy and precision, synchronization of the optical and mechanical scanning to optimize alignment among and registration of strips, and stitching of image strips in parallel with acquisition using custom software.
Materials and Methods
Our previous mosaicing method required stitching square-shaped images, with aspect ratio, in two dimensions, as shown in Fig. 1(a). The newly developed method acquires rectangular-shaped long strips of images,20 with aspect ratio for a 10-mm long strip, and stitches along the length of the strip as shown in Fig. 1(b). The elimination of a stitching dimension reduces the processing time and the artifacts due to illumination fall-off by half.
The strip mosaicing scheme, with combined optical and mechanical scanning, is shown in Fig. 2. The fast optical scanner produces a scan-line (in the direction) that defines the width of a strip. Scanning the length of the strip (in the direction) is performed by translating the stage in the direction orthogonal to the line. When the acquisition of a strip image is completed (“forward scan” in direction), the stage moves in the direction a distance equal to or less than the width of the strip. Then the stage moves along the strip length but in the opposite direction (“return scan”), and another strip image is acquired. This process of acquiring strip images, sequentially in opposite directions (forward scan followed by return scan) is repeated until the entire tissue is imaged. The aspect ratio of a strip is restricted only by the mechanical limits of the scanning stage.
Our system is based on a modified breadboard version of a commercial confocal microscope (Vivascope 2000, Lucid Inc, Rochester, NY). Details are available in our earlier papers.1314.–15,18,19 The microscope is set up to acquire images in fluorescence. The microscope is in an inverted configuration designed to image excised or biopsied tissue ex vivo. The illumination is from a 488-nm laser that delivers on the tissue. A 24-sided polygonal mirror scans the laser in the direction sweeping a line at through a , 0.75 numerical aperture (NA) water immersion objective lens (StableView, Lucid Inc.) The length of the scanned line, which defines the width of the strip, is 485 μm. The theoretical lateral resolution, according to the Rayleigh criterion, is 0.4 μm (Airy radius) and the expected diffraction-limited optical sectioning is 2.3 μm. The objective lens is custom-designed to image through a 1-mm-thick glass slide. In normal operation, this line is optically scanned in the orthogonal direction ( direction) by a galvanometrically driven mirror to produce a square image. To acquire a long rectangular-shaped strip image, we center and lock the position of the stop the galvanometric scanner.
The mechanical specifications of the translation stage, such as straightness, flatness, repeatability and accuracy are important factors for long strip scans. We use a high-quality dual-axis stage (BioPrecision2, Ludl Electronics Products, Hawthorne, NY) to mechanically translate the tissue with respect to the objective lens. The straightness and flatness of this stage is . Therefore, for a translation of 10-mm travel, the maximum deviation in straightness is 0.8 μm between any two strips. This is well within the overlap between any two strips. The maximum deviation in flatness is also within the optical sectioning of 2.3 μm.
The translation stage is equipped with a position encoder (MAC5000, Ludl). The encoder outputs are sent to a fast data acquisition card for synchronization in real time. The commands to initiate and control the movement of stages are sent to the controller through a USB bus.
Synchronization of strips in the y direction
In our earlier configuration,20 the translation stage mechanical scan (slow scan in direction) was driven by a stepper-motor without a position encoder. We accomplished the synchronization of the polygon driven optical scan (fast scan in direction) and the translation stage mechanical scan (slow scan in direction) by counting the steps from the stepper-motor. This method turned out to be unreliable. We observed mismatch of up to 75 lines, between any two strips, due to missing steps in the motors and the resulting lack of position accuracy. This mismatch was corrected with software during stitching of the strips but at the expense of more computer processing and increased time for creating mosaics. However, the current position synchronization in the direction reduces the work of the stitching algorithm, thus reducing the time required to complete the mosaic. The synchronization mechanism that we report in this paper ensures that the theoretical maximum mismatch between any two strips is one line in the direction. Figure 3 shows the synchronizing scheme for the strips.
Tissue flattening and stage leveling
In order to image a large area, the surface of the tissue is scanned in a two-dimensional (2-D) “image plane,” which is perpendicular to the optical axis of the objective lens in the microscope (Fig. 4). However, surgically excised tissue has a three-dimensional (3-D) topography with varying shape and size. Furthermore, the tissue has variable mechanical compliance; for example, skin tissue is relatively stiff whereas breast tissue is soft and gel-like. This requires that the tissue surface be flattened onto a 2-D plane that conforms to the image plane of the microscope [Fig. 4(c)]. For this purpose we designed a tissue flattening and mounting device, as shown in Fig. 4(a).
We use a rubber bladder filled with water to flatten the tissue onto a 1-mm-thick BK7 window inside a cassette Fig. 4(a). To flatten the desired (lower) surface, the user places the tissue at the center of the cassette, the water-bladder on top and closes the cassette cover. Then the tissue-cassette is placed on the tip-tilt plate, secured with two spring-loaded clips and placed on the microscope.
Acquisition of a large number of images over a large area at constant depth requires the imaging and scanning planes to be coincident. This is obtained by adjusting the tip-tilt plate with spring-loaded thumbscrews [Fig. 4(b)]. The three thumbscrews are placed at right angles to each other, and tip/tilt can be adjusted by initially turning either thumbscrew A or C [Fig. 4(a)], while observing images of the glass window at four corners. The image of the window may be either a circular spot when focused on the surface or a ring due to field curvature when focused past the glass. This adjustment is performed in an iterative manner until the images at the four corners appear identical to each other. This device and the alignment method allow large tissues to be matched to the image plane of the microscope, and enables mosaicing over large areas. The mounting of the tissue cassette and the adjustment of the tip-tilt are robust enough to allow repeatable operation during extended periods without the need to realign.
Discarded excisions skin and breast tissues were obtained under two IRB-approved protocols. The Dermatology Service provided discarded tissue from Mohs surgery, and breast tissue was obtained from The Evelyn H. Lauder Breast Center at Memorial Sloan-Kettering Cancer Center (MSKCC). Details about collection and handling of the tissues from Mohs surgery and staining methods have been extensively described in our earlier papers.14,21 The staining of nuclear morphology is accomplished by soaking the tissue in 0.6-mM acridine orange for 30 s for skin tissue and 1 min for breast tissue, followed by rinsing the excess with isotonic phosphate buffered saline solution.
Acquisition of Mosaics
For acquisition of mosaics, the resolution is set in our acquisition system to the level of nuclear and other morphologic detail needed to match standard histology. While relatively lower resolution may be adequate for imaging skin tissue, higher resolution is needed to analyze breast tissue.
The images are acquired in LabView and displayed in 8-bits. The 8-bit pixel depth was chosen because of our previous experience and for practical reasons. Clinical results from our previous studies18,19 have validated that 8-bits are sufficient for this particular application. Our Mohs surgeons found the quality of the resulting images to be clinically acceptable and were able to evaluate mosaics, in a manner similar to their reading of histopathology, with sensitivity of 96.6% and specificity of 89.2%. Another consideration is that 8-bit mosaics of large areas produce images of 300 to 400 MB. Increasing the depth to 16-bits would double their size making them difficult to manipulate. Such large image files pose practical challenges for subsequent processing and storing. This is further addressed in Sec. 2.5 on display of mosaics. The fluorescence images are captured with a fast DAQ card (PCI-6115 and Labview, National Instruments, Austin TX). The captured image strips are stitched with custom software developed by our group.
Mosaics of skin tissue
The acquisition time is limited by the speed of the optical (polygon) scanning rate of . The line is sampled with a 6.5-MHz pixel clock. Our theoretical lateral resolution (Airy radius) is . For the resulting spot diameter of , adequate sampling requires two lines per resolvable distance, which requires the speed of the mechanical stage to be less than . However, we scan our stage at choose to under-sample by a factor of about 4.7 to reduce the acquisition time. This results in a pixel size of in both and directions. This is adequate for the interpretation of images of skin by Mohs surgeons and pathologists, as was explained in our earlier paper.20 Once the scan is completed, the stage moves 400 μm laterally, leaving approximately a 19% overlap between any two strips. We repeat this process until the tissue is fully scanned.
Mosaics of breast tissue
The evaluation of breast tissue demands higher lateral resolution. To achieve a higher resolution, we slowed the polygon scan speed to 6.5 kHz, while holding sampling pixel clock at 6.5 MHz, thus slowing the acquisition time of a strip. To maintain adequate sampling, we must scan the mechanical stage at . However, we move the stage at , which results in under-sampling of about 3.8. At this stage speed, the resulting pixel size is in both and directions. The lateral move of the stage is 480 μm.
Stitching of Image Strips
A stitching algorithm was written in Matlab (R2011a, Mathworks, Inc.) to automatically register image strips in the order in which they are collected. The stitching algorithm has three parts: a registration algorithm, an intensity fall-off correction, and blending in the overlap region between the two strips.
The strip-scanning method, as compared with the traditional mosaicing method, simplifies registration of images, as each image needs to be registered to only one neighbor instead of two. Strips are registered pair-wise using a phase correlation method chosen for speed and ease of implementation, as it is easily computed by fast Fourier transform (FFT). Computing the phase correlation between the two images produces a 2-D matrix of correlation values where the coordinate of each value in the correlation matrix corresponds to a possible offset between the two images. The correlation value is maximal at the coordinate at which the same features in the two images overlap. But, because the correlation of two rectangular functions (such as two images) is a triangle function, the correlation is biased toward solutions near zero offset between the two images (the peak of the triangle). The bias is removed in the direction by dividing the phase correlation matrix by a triangle function. The removal of the bias emphasizes the noise near the edges of the correlation matrix, where solutions of maximal offset between the two images lie. Solutions of maximal offset correspond to no overlap between the images. Because we expect the images to overlap by an amount that is set by the stage motors, we can exclude solutions outside of the expected range. Therefore coordinates in the noisy region near the edges of the correlation matrix are excluded from the search region. In the direction, we expect solutions near zero offset, and therefore the removal of the bias was determined to be unnecessary along the direction. The solution for offset and offset between the two images is found by determining the maximum of the correlation matrix after the bias has been removed and the edges of the matrix have been excluded from the search region.
Performing a 2-D FFT on very large images requires a large amount of memory. In practice, only a small section of the strip is required to perform registration, as long as the chosen length of the section is greater than the offset between the two strips. The chosen strip section must contain features to register. In our setup, the tissue lies in the center of the glass window. Hence, a section that is 10 to 30% the length of the strip is chosen from the center region of each strip. This also serves the purpose of restricting the possible offset solutions to those that we know are most likely to be a good match. The search region is further restricted to coordinates near that expected from our knowledge of the stage motion. Generally, an apodizing window function is applied to an image before performing a phase-correlation algorithm to reduce high-frequency edge effects. However, in this case, the illumination fall-off across the strips serves this purpose.
The illumination fall-off is corrected after the registration algorithm is applied. Because the optical scanning through the objective lens is in one dimension only, the illumination fall-off is along only the direction of each strip. The fall-off is corrected by averaging each strip along its length, normalizing, and dividing each line in the strip by the normalized average fall-off profile of the strip.
In the overlap region between strips, the strips are blended by a weighted average of the overlapping pixels determined by the pixel distance to the edge of the strip. Pixels close to the edge of the strip are weighted less than pixels farther away. The result is a seamless mosaic, with more weight given to pixels near the center of the strip and less weight given to pixels near the edges of the strip.
Display of Mosaics
The mosaics are displayed in 8-bits on a large monitor with (30-in. flat-screen Dell 222–7175, with a GeForce 8800 GTS video card).When histopathology is examined with a bright field microscope, using, typically, a , 0.08NA objective lens with white light illumination, the observed field of view (FOV) is 10 mm with resolution. This FOV consists of approximately assuming one pixel per one optical resolution point. Our display matches this standard -view of histopathology. Zooming allows one to view submosaics at full resolution, mimicking the pathologist’s ability to switch to higher magnifications when necessary. Our Matlab program can easily manage the manipulation and display of mosaics of FOV . However, for larger mosaics, the program exhausts allowed allocated memory in our current system. (Potential future implementation at the bedside in, say, diverse and low-resource settings may require alternative practical and lower-cost approaches. One approach, for example, is scaling large mosaics to smaller size, while maintaining pathology-equivalent , which results in final size of 3 to 4 MB. This approach has been validated for adequate clinical quality and sensitivity and specificity by our Mohs surgeons.18,19
Figures 5Fig. 6Fig. 7Fig. 8Fig. 9 to 10 show examples of strip mosaics of varying sizes, with their corresponding hematoxylin and eosin (H&E)-stained pathology. The mosaics are shown in fluorescence contrast, following staining with acridine orange. The resolution and nuclear-level morphology in the mosaics may not be easily appreciated in these figures, but is more clearly seen when viewed on our large monitor with adequate pixelation. In these figures, we include inset areas that are displayed with zoom to show the resolution and level of detail.
Skin Tissue from Mohs Surgery
Figure 5 shows a strip mosaic of a skin excision from Mohs surgery. The measured time for the mosaic, consisting of 24-strip images, was about 3 min. This mosaic was created before we implemented simultaneous stitching with the acquisition. The strips were stitched after acquisition was completed, adding approximately an extra 96 s to complete the mosaic.
Nuclear detail and morphology is seen in the magnified insets A, B, and C. The morphologic features in the mosaics compare well to those in the corresponding pathology (Fig. 6) for both basal cell carcinomas and normal skin. However, a perfect one-to-one correlation of individual features is not always expected (nor desired by Mohs surgeons), because tissue is pliant and thus susceptible to distortions introduced during the fixturing for mosaicing or processing for frozen sections. As a result, the pathology is usually of a 5-μm-thin tissue section that may be adjacent to, but not exactly of, the imaged surface.
Figure 7 is also a strip mosaic of a skin excision from Mohs surgery. This mosaic is consisting of 17-strip images. The image strips were stitched simultaneously with acquisition, and the mosaic was completed in 55 s [The simultaneous stitching eliminating the extra time that was otherwise required for some of our earlier mosaics (Fig. 8)]. The nuclear detail and morphology is seen in the magnified insets A, B, and C. The morphologic features in the mosaics compare well to corresponding pathology (Fig. 8).
Breast Tissue from Breast Cancer Surgery
Figure 9 is a strip mosaic of breast tissue from a lumpectomy. The actual size of the tissue is about . A step-shaped artifact can be seen at the upper and lower edges of the mosaic (Fig. 9). This was due to a malfunction in the tissue translation stage. [The malfunction manifested as a loss of position in the scan along the strip ( direction) while stepping in the orthogonal direction ( direction.)] Thirty 3.5-cm-long image strips were acquired in 13 min and stitched after acquisition. For this mosaic, we can only report the acquisition time due to hardware limitation in the present implementation. The scan of a such large area requires about 60 image strips. Our stitching program (written in Matlab) runs out of allowed allocated memory during attempts to stitch all the 2.5-cm long strips. Each strip is a factor of 2.5 longer than our usual 10-mm strip. In this mosaic we first stitched four 2.5-cm strips and used ImageJ to merge 4-strip images together to create the mosaic. In the future, this hurdle may be overcome by implementing a graphic processing unit (GPU) dedicated for stitching and display. Since the strips are being stitched during acquisition we expect the full mosaic could be produced in 13 min with a dedicated GPU (plus the time to scan and stitch the last strip).
Within the mosaic, regions of interest can be identified. The central bright-appearing region is tumor and the outlying darker-appearing regions are primarily fat and connective tissue. Upon closer inspection, cellular and increased nuclear density is visible. For example, inset B of Fig. 9 shows a duct. A normal duct is made up of a double layer of cells surrounding an empty lumen. In this case, the normal growth of the epithelial cells has been disrupted and while the overall shape resembles a duct, the cellular architecture shows that this is no longer a typical cellular pattern. Inset C of Fig. 9 shows proliferation of cell growth in a duct. Instead of the normal double-cell layer, cells have grown inward to the lumen. The cells are contained within the boundaries of the duct, suggesting a ductal carcinoma in situ. Inset A of Fig. 9 shows invasive carcinoma in which the cancer cells have invaded the surrounding fat cells. The fat cells are large and appear dark. The tumor cells are small with bright nuclei.
Summary and Discussion
Our previous approaches for confocal mosaicing microscopy proved the feasibility of imaging nuclear and cellular morphology and tumor margins in large areas of freshly excised tissue.1314.–15,21,22 Two Mohs surgeons demonstrated their ability to read mosaics in a manner similar to their examination of pathology, and detect basal cell carcinoma margins in Mohs surgical excisions with high sensitivity and specificity.18,19 Although mosaicing could be performed in several minutes, the speed was still too slow to enable routine and practical use in Mohs and other surgical settings. To overcome this barrier and address the request for shorter times of surgeons, we designed a faster approach called “strip mosaicing.” The initial feasibility was reported last year.20 In this paper, we presented further advances in electronics, mechanics, custom software, and the overall integration of hardware and software into an approach that offers further improvement in speed.
Strip mosaicing confocal microscopy offers an imaging technology platform for rapid detection of tumor margins directly in fresh tissue during surgery. Large amounts of tissue may be examined quickly enough to be of practical use in diverse surgical settings. The imaging may potentially be developed into an adjunct for pathology, to enable more complete and accurate removal of tumor. Further improvements in strip mosaicing may be achieved in several ways:
1. Increasing the speed of the polygon scanner (which is currently the limiting factor).
2. Employing other faster scanning methods.
3. Increasing the field-of-view while accepting lower but clinically acceptable resolution.
4. Automating the process for tissue mounting, flattening and leveling the imaging surface.
As shown in this paper, tissues from Mohs surgery and breast tumors are good tests to demonstrate strip mosaicing indifferent surgical settings. This approach may also find applications in other tissues, and not only in surgical settings but also in clinics for rapid screening of biopsies. Each application will, of course, present its own challenges in terms of the mechanical properties (shape, size, topography, compliance) of the tissue to be imaged, availability and efficacy of contrast agents, speed requirement and tissue fixturing for translation to the bedside. Currently there are many mosaicing approaches being developed for microscopy of processed tissue23220.127.116.11.18.104.22.168.32.–33 as well as for in vivo.3422.214.171.124.–39 Our instrumentation design will help in developing those approaches.
Beyond imaging excised tissue, a similar approach may be possible on patients to delineate tumor margins in vivo, either preoperatively or intraoperatively in surgical wounds. Preliminary feasibility of such techniques has been reported for mosaicing on skin in vivo17,34126.96.36.199.–39 and also in shave-biopsied wounds in which residual tumor (basal and squamous cell carcinoma) margins were delineated.40 In the future, an approach that involves mosaicing both on excised tissue at the bedside and intra-operatively on the patient may prove useful for detecting tumor margins in a rapid, efficient, and cost-effective manner.
The authors gratefully acknowledge support from NIH grant R01EB012466 from NIBIB’s Image Guided Interventions Program (Program Director Dr. Steve Krosnick). We thank Drs. Kishwer Nehal and Erica Lee for supplying discarded tissue from Mohs surgery, and Drs. Melissa Murray and Monica Morrow for supplying tissue from breast cancer surgeries (under IRB-approved protocols), and, most importantly, for their enthusiastic collaborative involvement with image analysis and correlation to pathology. We are grateful to Dr. Chris Glazowski for help with assembly and alignment of the microscope, Bill Fox and Scott Grodevant at Lucid Inc. and our machine shop at MSKCC for technical support.