There are new cases of cancer each year, and more than 80% of these patients will require surgery, some several times.1 Surgery cures of all patients with cancer.2,3 To cure a cancer patient by surgery, the surgeon must remove the entire tumor at the time of the surgery. Unfortunately, up to 39% of the patients who undergo surgery leave the operating room without a complete resection due to positive or close margins.2,4,5 It has been reported that a complete resection is the single most important predictor of patient survival for almost all solid cancers.6 In breast, head and neck (H&N), lung, colon, and pancreatic cancers, complete resection is associated with a threefold to fivefold improvement in the patient survival compared to a partial or incomplete resection. 78.9.–10
For cancer margin assessment, various methods have been used or are under development in order to improve the tumor resection during surgery. The visual appearance and palpation are often used by a surgeon to differentiate between malignant and normal tissue.11 However, this visual assessment is subjective. Intraoperative, frozen margin evaluation is commonly used to optimize surgical margin delineation at the initial surgery.12 Small samples from the surgical bed are selected to evaluate the presence or absence of residual cancer.13 However, intraoperative, frozen section diagnosis may suffer from errors that occur during sampling and histological interpretation. In addition, histological processing can take time,14 which is labor-intensive and prolongs the surgery time. Fluorescence-guided imaging used to navigate cancer resection has been shown to improve the number of complete resections as well as the progression-free survival.1522.214.171.124.20.–21 In most cases, fluorescence-based approaches require the injection of a fluorescence contrast agent. There are clinical needs to develop label-free imaging technology and quantification methods to aid the decision-making during image-guided surgery.
Hyperspectral imaging (HSI), originated from the remote sensing field,22 has emerged as a relatively new imaging modality for medical applications.23 This label-free imaging technology does not require a contrast agent and offers great potential for objective assessment of cancer margins. Light delivered to the biological tissue undergoes multiple scattering due to the inhomogeneity of biological structures and absorption primarily in hemoglobin, melanin, and water as it propagates through tissue.24,25 The absorption, fluorescence, and scattering characteristics of tissue change with the progression of diseases.26 Therefore, the reflected, fluorescent, and transmitted lights from tissue, where are captured by HSI, carry quantitative diagnostic information regarding tissue pathology.2627.28.–29 Spatially resolved spectra obtained by HSI provide diagnostic information about the tissue physiology, morphology, and composition. Recent advancements of hyperspectral cameras, image analysis methods, and computational power make it possible for many exciting applications of HSI, such as cancer detection and image-guided surgery.23
Previously, our group has developed a pipeline of machine-learning-based quantification methods for hyperspectral data, including image preprocessing, feature extraction and selection, and image classification, and validated these approaches in multiple preclinical animal models for both noninvasive cancer detection3031.32.–33 and surgical guidance.34 To simulate the characteristics of surgical images, our group34 developed a framework of hyperspectral image processing and quantification and validated the method for cancer detection during animal surgery. These preclinical studies have demonstrated that HSI has great potential to be used as a diagnostic tool for cancer detection. To translate HSI into the clinic, we designed a plot study to image surgical specimen of H&N cancer patients and implemented and validated a pipeline of quantification method to differentiate tumor from normal tissue in HSI data of human patients. The cancer detection results of HSI were compared with histopathology to determine the sensitivity and specificity.
The preliminary result of this pilot study was presented at the 2017 SPIE Photonics West Conference “Advanced Biomedical and Clinical Diagnostic and Surgical Guidance Systems XV” and was selected by the conference chairperson for submitting a full paper to the Journal of Biomedical Optics.35 In this paper, we extended the SPIE paper and added more results and discussions. This pilot study represented an important first step toward translating label-free HSI into the clinic for assessing the tumor margins of H&N cancer tissue from human patients.
Methods and Materials
Overview of the Study Design
Figure 1 shows the study design for the HSI experiment on surgical specimens of H&N cancer patients. Before surgery, consent was obtained from the patient, and clinical information was collected. During surgery, three types of tissue, i.e., clinically visible tumor, normal tissue, and tumor with adjacent normal tissue, were collected and prepared for HSI, autofluorescence imaging, and fluorescence imaging with vital dyes, i.e., 2-deoxy-2-[(7-nitro-2,1,3-benzoxadiazol-4-yl)amino]-D-glucose (2-NBDG) and proflavine. The tissues were then processed histologically, and the pathological images were digitized and analyzed by a pathologist for validation.
Hyperspectral Imaging System
A Maestro (PerkinElmer Inc., Waltham, Massachusetts) imaging system was used to acquire the hyperspectral dataset. This is a wavelength-scanning system consisting of a xenon light source, a solid-state liquid crystal filter, and a 12-bit high-resolution charge-coupled device. Details regarding this system have been described in previously published papers.36,37 This system is capable of obtaining reflectance images over the range of 450 to 950 nm with a 2-nm increment, as well as fluorescence images under different excitation light sources.38
Surgical Specimen Collection
H&N cancer patients who underwent surgery at Emory University Hospitals Midtown were recruited into the study. All tissues were collected under the clinical protocol approved by the Institutional Review Board of Emory University. During surgery, fresh surgical specimens were collected and sent to the Pathology Department for cancer assessment. Three tissue samples, i.e., (i) clinically visible tumor, (ii) surrounding normal tissue, and (iii) tumor with adjacent normal tissue, were collected from the main specimen of each consented patient. These specimens were rinsed with cold phosphate-buffered saline (PBS) to remove excess blood on tissue.
Hyperspectral Image Acquisition and Quantification
Fresh surgical specimens were scanned with HSI in the following steps: (1) acquire white and dark reference hypercube before tissue imaging. White reference image cubes are acquired by placing a standard white reference board in the field of view. The dark reference cubes are acquired by keeping the camera shutter closed, (2) acquire reflectance hyperspectral images of the specimen from 450 to 900 nm with 5-nm intervals, (3) acquire autofluorescence images of the specimen with blue excitation at 455- and 490-nm long-pass emission filter. (4) Acquire fluorescence imaging with 2-NBDG (Cayman Chemical, Ann Arbor, Michigan) using blue excitation and 490 long-pass filter. For 2-NBDG imaging, the specimen was washed once after incubation of tissue in a solution of 2-NBDG in PBS for 20 min at 37°C as described in Ref. 39 and (5) similarly, acquire fluorescence imaging with proflavine (Sigma Aldrich, St. Louis, Missouri) using the same imaging setting as described in step 4). After 2-NDBG imaging, the specimen was washed once and then incubated in a 0.01% w/v solution of proflavine in PBS for 2 min at the room temperature for proflavine imaging.
Figure 2 showed the flowchart of the machine-learning-based quantification approach for hyperspectral images. In this pilot study, the steps of preprocessing, feature extraction, and image classification were implemented and validated for cancer detection in surgical specimens. To establish the histopathology gold standard, the tissue samples were histologically processed with hematoxylin and eosin (H&E) staining and pathology slides were digitally scanned, and a clinically experienced pathologist outlined the tumor border on the digitized images for the validation of the HSI classification. HSI images, autofluorescence images, and fluorescence images were manually aligned with the H&E images to map the tumor region with a software system (Analyze, AnalyzeDirect, Inc.).
Hyperspectral Data Normalization
The purpose of data normalization was to remove the spectral nonuniformity of the illumination device and the influence of dark current. The raw data are normalized using the following equation:
Glare Detection and Removal
Glare regions are formed due to specular reflection from the moist tissue surface and do not contain useful diagnostic information regarding the tissue. Similar to the method we reported in Ref. 34, the glare detection method includes the following steps: (1) calculate the total reflectance of each pixel within a hypercube to form one reflectance image; glare pixels have higher total reflectance than normal pixels. (2) Compute the intensity histogram of this image, fit the histogram with a log logistic distribution, and then experimentally identify a threshold that separates glare and nonglare pixels.
Hyperspectral Image Classification
We analyze the spectral data in order to classify each pixel into normal or cancer tissue. To reduce the computational time without reducing the accuracy, spectral curves were averaged in nonoverlapping blocks of in order to yield a spectral signature per block. All of the spectral information available in the hyperspectral data was utilized. Blocks containing glare pixels were excluded from the classification process. Each block was assigned a label as cancerous or normal.
For each patient, we used the images of the tumor and normal tissue to train the classification algorithms and then used the tumor with adjacent normal tissue to test the performance of the classification model. In other words, the classification method built the training model using the spectral features extracted from the tumor and normal tissue, and the model was then evaluated on the tumor–normal interface tissue of the same patient. We applied ensemble linear discriminant analysis as the classifier using MATLAB® (MathWorks, Natick, Massachusetts).40
We used the pathology images of the same surgical specimen to validate the cancer detection using hyperspectral image classification. On the digitized, H&E-stained, pathology slides, the tumor margin was outlined by an experienced pathologist specialized in H&N cancer. To reasonably assess the performance of the classification, we chose the regions of interest where the tumor or normal tissue had been histopathologically confirmed by the pathologist.
The sensitivity and specificity of the classifiers for each patient were calculated based on the number of correctly classified tumor and normal pixels/blocks of all the specimens belonging to this patient. We also calculated how many normal pixels/blocks were correctly classified for a normal specimen and how many tumor pixels/blocks were correctly classified for a tumor specimen as well as the sensitivity and specificity on a tumor–normal interface specimen for each patient. We evaluated the performance of the hyperspectral image classification using the areas under the curve (AUC), accuracy, sensitivity, and specificity, as defined in the following equations [true negative (TN), true positive (TP), false positive (FP), and false negative (FN)]:
Fresh surgical tissue specimens were collected from 16 H&N cancer patients. The characteristics of these patients were listed in Table 1. These patients included seven with oral cancer, one with maxillary sinus cancer, five with thyroid cancer, one with parotid cancer, and two with larynx cancer. As described above, we collected three types of tissue specimens from each human patient, which included (i) clinically visible tumor tissue without necrosis, (ii) normal tissue, and (iii) tumor with adjacent normal tissue at the tumor–normal interface. Figure 3 shows the three tissue specimens and their corresponding histological slides.
|Patient||Age||Gender||Race||Tumor site||Histologic type|
|1||55||F||White||Tongue||Squamous cell carcinoma|
|6||51||F||White||Floor of mouth|
|7||57||M||White||Floor of mouth|
|9||69||M||African American||Thyroid||Papillary thyroid carcinoma|
|14||39||M||African American||Parotid||Pleomorphic adenoma|
Note: Patients 1 to 8, 15, and 16: squamous cell carcinoma; patients 9 to 13: papillary thyroid carcinoma.
Using the reflectance spectra from HSI, the HSI method was able to distinguish between cancer and normal tissue of the oral cavity with an average accuracy of , sensitivity of , and specificity of . For tissue specimens from the thyroid, the method achieved an average accuracy of , sensitivity of , and specificity of . As shown in Table 2, HSI outperformed autofluorescence imaging, 2-NBDG, and proflavine fluorescence imaging for both cancer sites.
Classification performance of HSI, autofluorescence imaging, and fluorescence imaging with 2-NBDG and proflavine.
|Cancer site||Imaging method||AUC||Accuracy (%)||Sensitivity (%)||Specificity (%)|
Figure 4 shows the photographs of the tumor and normal tissue as well as the tumor with adjacent normal tissue for a typical case. The three types of tissue demonstrate different spectral curves. The tumor margin, as assessed by the classification method, was close to that of the histological image outlined by the pathologist.
In this study, we reported regarding automated tissue classification methods that use the spectra from 450 to 900 nm to extract diagnostic information. Each hyperspectral image contains more than two million reflectance spectral signatures. The reflectance spectra capture the alteration of absorption and scattering properties of the tissue associated with malignant transformation. Molecular fingerprinting based on inverse modeling of reflectance spectra may shed new light on our understanding of cancer biology.
Although frozen section diagnosis is commonly used to guide surgical resection during surgery, it only samples a small portion of tissue in the resection area, which may lead to underestimation and does not guarantee margin-negative resection. In addition, this procedure is time-consuming and labor-intensive. HSI is a wide-field imaging modality that can cover a large field of view and can, therefore, provide rapid assessment of complete resection margins.
In this surgical specimen study, the label-free, HSI was superior to autofluorescence imaging or fluorescence imaging with vital dye (2-NBDG or proflavine) for the detection of H&N cancer. A recent study showed that wide-field fluorescence imaging with 2-NBDG can accurately distinguish the pathologically normal and abnormal biopsy tissue of H&N cancer patients.41 Proflavine has also been used for distinguishing between normal and neoplastic mucosa in the H&N.42 We previously demonstrated the utility of HSI for H&N cancer detection in a subcutaneous cancer animal model31,34 and a chemically induced oral cancer model.43 One important advantage of HSI is that it does not require the use of an exogenous contrast agent. Therefore, this noninvasive imaging technology can be rapidly translated from ex vivo tissue specimens to in vivo human studies, such as in clinical trials of hyperspectral, image-guided surgery.
In this study, we used the tumor and normal tissue from the same patient to train the classification and then to classify the tumor tissue with adjacent normal tissue. This approach provides reliable results and high accuracy for differentiating tumor from normal tissue. This approach is useful during surgery as this technology is helpful to the surgeon for differentiating the tumor margin while the clinician normally is aware of the tumor core but is not certain regarding the boundary of the tumor. In the future, we will test another approach that uses different patients’ data to train the classification and then test the method on a new patient. This requires a large database and we are currently collecting tissue from more patients. The combination of the two approaches may be able to provide a useful tool for the surgeon to achieve complete resection and thus improve both the patient survival and outcome.
HSI is an emerging imaging modality for medical applications. This label-free, HSI technology does not require a contrast agent and offers great potential for cancer detection and image-guided surgery. Hyperspectral large data contain both spatial and spectral information. Our hyperspectral image quantification tools are able to distinguish cancer from normal tissue in fresh surgical specimens of H&N cancer patients. Further development of the HSI technology is warranted for its application in image-guided surgery.
The authors have no relevant financial interests in this paper and no potential conflicts of interest to disclose.
This research was supported in part by the National Institutes of Health Grants (Nos. CA176684, R01CA156775, and CA204254) and by Developmental Funds from the Winship Cancer Institute of Emory University under Award No. P30CA138292. The authors would like to thank the team at Emory University Hospital Midtown, including Andrew Balicki, Jacqueline Ernst, Tara Meade, Dana Uesry, and Mark Mainiero, for their help in collecting fresh tissue specimens.
R. Sullivan et al., “Global cancer surgery: delivering safe, affordable, and timely cancer surgery,” Lancet Oncol. 16(11), 1193–1224 (2015).http://dx.doi.org/10.1016/S1470-2045(15)00223-5Google Scholar
S. Singhal, S. M. Nie and M. D. Wang, “Nanotechnology applications in surgical oncology,” Annu. Rev. Med. 61, 359–373 (2010).http://dx.doi.org/10.1146/annurev.med.60.052907.094936Google Scholar
A. M. De Grand and J. V. Frangioni, “An operational near-infrared fluorescence imaging system prototype for large animal surgery,” Technol. Cancer Res. Treat. 2(6), 553–562 (2003).http://dx.doi.org/10.1177/153303460300200607Google Scholar
Q. T. Nguyen and R. Y. Tsien, “Fluorescence-guided surgery with live molecular navigation—a new cutting edge,” Nat. Rev. Cancer 13(9), 653–662 (2013).NRCAC41474-175Xhttp://dx.doi.org/10.1038/nrc3566Google Scholar
C. Gebitekin et al., “Fate of patients with residual tumour at the bronchial resection margin,” Eur. J. Cardio-Thorac. Surg. 8(7), 339–342 (1994).EJCDBR0301-4711http://dx.doi.org/10.1016/1010-7940(94)90025-6Google Scholar
F. Meric et al., “Positive surgical margins and ipsilateral breast tumor recurrence predict disease-specific survival after breast-conserving therapy,” Cancer 97(4), 926–933 (2003).CANCAR0008-543Xhttp://dx.doi.org/10.1002/(ISSN)1097-0142Google Scholar
W. Sienel et al., “Frequency of local recurrence following segmentectomy of stage IA non-small cell lung cancer is influenced by segment localisation and width of resection margins—implications for patient selection for segmentectomy,” Eur. J. Cardio-Thorac. Surg. 31(3), 522–528 (2007).EJCDBR0301-4711http://dx.doi.org/10.1016/j.ejcts.2006.12.018Google Scholar
T. Karni et al., “A device for real-time, intraoperative margin assessment in breast-conservation surgery,” Am. J. Surg. 194(4), 467–473 (2007).http://dx.doi.org/10.1016/j.amjsurg.2007.06.013Google Scholar
J. P. Neoptolemos et al., “Influence of resection margins on survival for patients with pancreatic cancer treated by adjuvant chemoradiation and/or chemotherapy in the ESPAC-1 randomized controlled trial,” Ann. Surg. 234(6), 758–768 (2001).http://dx.doi.org/10.1097/00000658-200112000-00007Google Scholar
Q. T. Nguyen et al., “Surgery with molecular fluorescence imaging using activatable cell-penetrating peptides decreases residual cancer and improves survival,” Proc. Natl. Acad. Sci. U. S. A. 107(9), 4317–4322 (2010).http://dx.doi.org/10.1073/pnas.0910261107Google Scholar
J. D. Meier, D. A. Oliver and M. A. Varvares, “Surgical margin determination in head and neck oncology: current clinical practice. The results of an International American Head and Neck Society Member Survey,” Head Neck 27(11), 952–958 (2005).http://dx.doi.org/10.1002/(ISSN)1097-0347Google Scholar
R. F. Gandour-Edwards, P. J. Donald and D. A. Wiese, “Accuracy of intraoperative frozen section diagnosis in head and neck surgery: experience at a university medical center,” Head Neck 15(1), 33–38 (1993).http://dx.doi.org/10.1002/(ISSN)1097-0347Google Scholar
J. M. Warram et al., “A ratiometric threshold for determining presence of cancer during fluorescence-guided surgery,” J. Surg. Oncol. 112(1), 2–8 (2015).JSONAU0022-4790http://dx.doi.org/10.1002/jso.23946Google Scholar
E. L. Rosenthal et al., “Safety and tumor specificity of Cetuximab-IRDye800 for surgical navigation in head and neck cancer,” Clin. Cancer Res. 21(16), 3658–3666 (2015).http://dx.doi.org/10.1158/1078-0432.CCR-14-3284Google Scholar
C. Hirche et al., “An experimental study to evaluate the Fluobeam 800 imaging system for fluorescence-guided lymphatic imaging and sentinel node biopsy,” Surg. Innov. 20(5), 516–523 (2013).http://dx.doi.org/10.1177/1553350612468962Google Scholar
J. R. van der Vorst et al., “Intraoperative near-infrared fluorescence imaging of parathyroid adenomas with use of low-dose methylene blue,” Head Neck 36, 853–858 (2014).http://dx.doi.org/10.1002/hed.v36.6Google Scholar
B. E. Schaafsma et al., “Clinical trial of combined radio- and fluorescence-guided sentinel lymph node biopsy in breast cancer,” Br. J. Surg. 100(8), 1037–1044 (2013).http://dx.doi.org/10.1002/bjs.9159Google Scholar
Q. Tummers et al., “The value of intraoperative near-infrared fluorescence imaging based on enhanced permeability and retention of indocyanine green: feasibility and false-positives in ovarian cancer,” PLoS One 10(6), e0129766 (2015).POLNCL1932-6203http://dx.doi.org/10.1371/journal.pone.0129766Google Scholar
A. F. H. Goetz, “Three decades of hyperspectral remote sensing of the Earth: a personal view,” Remote Sens. Environ. 113(Suppl. 1), S5–S16 (2009).RSEEA70034-4257http://dx.doi.org/10.1016/j.rse.2007.12.014Google Scholar
L. V. Wang and H.-I. Wu, “Introduction,” in Biomedical Optics, pp. 1–15, John Wiley and Sons, Inc., Hoboken, New Jersey (2009).Google Scholar
B. Costas, P. Christos and E. George, “Multi/hyper-spectral imaging,” in Handbook of Biomedical Optics, pp. 131–164, CRC Press, Boca Raton, Florida (2011).Google Scholar
V. V. Tuchin and V. Tuchin, Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis, SPIE Press, Bellingham, Washington (2007).Google Scholar
D. G. Ferris et al., “Multimodal hyperspectral imaging for the noninvasive diagnosis of cervical neoplasia,” J. Lower Genital Tract Dis. 5(2), 65–72 (2001).http://dx.doi.org/10.1046/j.1526-0976.2001.005002065.xGoogle Scholar
M. C. Pierce et al., “Accuracy of in vivo multimodal optical imaging for detection of oral neoplasia,” Cancer Prev. Res. 5(6), 801–809 (2012).http://dx.doi.org/10.1158/1940-6207.CAPR-11-0555Google Scholar
H. Akbari et al., “Hyperspectral imaging and quantitative analysis for prostate cancer detection,” J. Biomed. Opt. 17(7), 076005 (2012).JBOPFO1083-3668http://dx.doi.org/10.1117/1.JBO.17.7.076005Google Scholar
G. Lu et al., “Spectral-spatial classification for noninvasive cancer detection using hyperspectral imaging,” J. Biomed. Opt. 19(10), 106004 (2014).JBOPFO1083-3668http://dx.doi.org/10.1117/1.JBO.19.10.106004Google Scholar
R. Pike et al., “A minimum spanning forest-based method for noninvasive cancer detection with hyperspectral imaging,” IEEE Trans. Biomed. Eng. 63(3), 653–663 (2016).IEBEAX0018-9294http://dx.doi.org/10.1109/TBME.2015.2468578Google Scholar
G. Lu et al., “Hyperspectral imaging of neoplastic progression in a mouse model of oral carcinogenesis,” Proc. SPIE 9788, 978812 (2016).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.2216553Google Scholar
G. Lu et al., “Framework for hyperspectral image processing and quantification for cancer detection during animal tumor surgery,” J. Biomed. Opt. 20(12), 126012 (2015).JBOPFO1083-3668http://dx.doi.org/10.1117/1.JBO.20.12.126012Google Scholar
B. Fei et al., “Tumor margin assessment of surgical tissue specimen of cancer patients using label-free hyperspectral imaging,” Proc. SPIE 10054, 100540E (2017).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.2249803Google Scholar
G. Lu et al., “Spectral-spatial classification using tensor modeling for cancer detection with hyperspectral imaging,” Proc. SPIE 9034, 903413 (2014).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.2043796Google Scholar
G. Lu et al., “Hyperspectral imaging for cancer surgical margin delineation: registration of hyperspectral and histological images,” Proc. SPIE 9036, 90360S (2014).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.2043805Google Scholar
L. V. Halig et al., “Biodistribution study of nanoparticle encapsulated photodynamic therapy drugs using multispectral imaging,” Proc. SPIE 8672, 867218 (2013).PSISDG0277-786Xhttp://dx.doi.org/10.1117/12.2006492Google Scholar
A. Hellebust et al., “Vital-dye-enhanced multimodal imaging of neoplastic progression in a mouse model of oral carcinogenesis,” J. Biomed. Opt. 18(12), 126017 (2013).JBOPFO1083-3668http://dx.doi.org/10.1117/1.JBO.18.12.126017Google Scholar
T. K. Ho, “The random subspace method for constructing decision forests,” IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998).ITPIDJ0162-8828http://dx.doi.org/10.1109/34.709601Google Scholar
Z. Luo et al., “Widefield optical imaging of changes in uptake of glucose and tissue extracellular pH in head and neck cancer,” Cancer Prev. Res. 7(10), 1035–1044 (2014).http://dx.doi.org/10.1158/1940-6207.CAPR-14-0097Google Scholar
P. Vila et al., “Discrimination of normal and neoplastic mucosa with a high-resolution microendoscope (HRME) in head and neck cancer,” Ann. Surg. Oncol. 19(11), 3534–3539 (2012).http://dx.doi.org/10.1245/s10434-012-2351-1Google Scholar
Baowei Fei is an associate professor in the Department of Radiology and Imaging Sciences, Emory University School of Medicine. He is also a primary faculty member in the Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology. He is a faculty member in the Department of Mathematics and Computer Sciences, Emory University. He is a Georgia Research Alliance Distinguished Cancer scientist, director of Quantitative Bioimaigng Laboratory (www.feilab.org), and director of Precision Imaging Research Program in the Department of Radiology and Imaging Sciences, Emory University School of Medicine.