Endometrial cancer (EC) is the most common gynecologic malignancy in the United States. Hormone therapies and hysterectomy are viable treatments for early-stage EC and atypical endometrial hyperplasia (AEH), a high-risk precursor to EC. Prediction of patient response to hormonal treatment is useful for patients to make treatment decisions. We have previously developed a mix-supervised model: a weakly supervised deep learning model for hormonal treatment response prediction based on pathologist-annotated AEH and EC regions on whole slide images of H&E stained slides. The reliance on pathologist annotation in applying the model to new cases is cumbersome and subject to inter-observer variability. In this study, we automate the task of ROI detection by developing a supervised deep learning model to detect AEH and EC regions. This model achieved a patch-wise AUROC performance of 0.974 (approximate 95% CI [0.972, 0.976]). The mixsupervised model yielded a patient-level AUROC of 0.76 (95% CI [0.59, 0.92]) with ROIs detected by our new model on a hold-out test set in the task of classifying patients into responders and non-responders. As a comparison, the original model as tested on pathologist-annotated ROIs achieved an AUROC of 0.80 with 95% CI [0.63, 0.95]. Our results demonstrate the potential of using weakly supervised deep learning and supervised ROI detection model for predicting hormonal treatment response in endometrial cancer patients.
The proper guidance of the epidural needle is crucial for safe and effective epidural anesthesia. In this research, we developed an innovative endoscopic system based on polarization-sensitive optical coherence tomography (PS-OCT). To assess its feasibility, we conducted experiments using ex vivo human epidural specimens. During the experiments, we imaged and analyzed different spinal tissue layers that the epidural needle passes through, including subcutaneous fat, supraspinous and interspinous ligament, ligamentum flavum, epidural space, dura, and spinal cord. Each of these tissue layers exhibited distinct imaging patterns. In addition, we used deep learning for automatic tissue recognition.
Lung transplantation is the last therapy option for patients with advanced lung diseases but the shortage of donor lungs currently is a large challenge. We used a polarization-sensitive optical coherence tomography (PS-OCT) system to obtain the distribution and quantification of alveolus and fibrous tissues. Our results showed that the distribution of alveolus and fibrosis was various at different locations on the lung. PS-OCT was able to provide effective quantifications of the alveolus density and size, and the fibrous tissues verified by the histology results. PS-OCT could serve as a promising tool for assessing the quality of donor lungs.
Kidney transplantation faces a worldwide shortage due to the lack of reliable assessment for screening qualified donor kidneys for transplantation. We evaluated the feasibility of using polarization-sensitive optical coherence tomography (PS-OCT) to provide a score map covering the entire surface of a kidney to evaluate the pre-transplantation kidney quality. Multiple histology staining and two-photon microscopy (TPM) were used to provide verification standards for microstructures, tissue distributions, and fibrosis in PS-OCT imaging. Our results indicated that PS-OCT was a reliable method for noninvasively imaging kidney microstructure and fibrosis matching the pretransplant scoring system for assessing the quality of pre-transplantation kidneys.
Early and accurate detection of renal tumor malignancy remains a critical challenge in clinical cancer diagnosis and treatment. Unfortunately, a third of all patients aren’t diagnosed until they have advanced disease. Percutaneous renal biopsy (PRB) followed by histopathology is the most commonly used surgical procedure for early kidney detection and diagnosis. However, PRB is challenging in precisely recognizing the tumor tissue and avoiding renal hemorrhage. In this project, we developed an endoscopic polarization-sensitive optical coherence tomography (PS-OCT) probe for PRB guidance. Deep-learning method was used to automate the tumor recognition procedure.
Fourier ptychography microscopy (FPM) is a computational imaging technique that enables high resolution and large FOV simultaneously. For FPM, multiplexed LED illumination can significantly improve the efficiency of image data acquisition at the cost of deteriorated quality in the reconstructed images. In this study, we aim to evaluate the imaging quality of multiplexed FPM with different illumination configurations. For this purpose, a prototype FPM microscope was developed, which was equipped with a 4×/0.1 NA objective lens. This prototype was used to test 1 LED conventional, 2 LED multiplexed, and 4 LED multiplexed FPM illumination configurations on a standard USAF 1951 resolution target and a cytology sample. Modulation transfer function (MTF) curves were generated from the reconstructed images to quantitatively compare the performance of different LED combinations. The results demonstrated that the resolution target image reconstructed using 1 LED illumination raw images can resolve up to 912.3 lp/mm, but it decreased to 812.7 lp/mm and 724.1 lp/mm when 2 LED and 4 LED illumination were adopted, respectively. The corresponding MTF curves indicate decreased contrast on most spatial frequencies when comparing reconstructed results between multiplexed (2/4 LED) and conventional illumination configurations. Accordingly, the quality of reconstructed clinical cytology sample images decreases as the number of LEDs per image increases. However, all of them have satisfactory quality for most clinical applications. This preliminary study provides useful information to facilitate the development of multiplexed illumination FPM imaging systems in the future.
PurposeEndometrial cancer (EC) is the most common gynecologic malignancy in the United States, and atypical endometrial hyperplasia (AEH) is considered a high-risk precursor to EC. Hormone therapies and hysterectomy are practical treatment options for AEH and early-stage EC. Some patients prefer hormone therapies for reasons such as fertility preservation or being poor surgical candidates. However, accurate prediction of an individual patient’s response to hormonal treatment would allow for personalized and potentially improved recommendations for these conditions. This study aims to explore the feasibility of using deep learning models on whole slide images (WSI) of endometrial tissue samples to predict the patient’s response to hormonal treatment.ApproachWe curated a clinical WSI dataset of 112 patients from two clinical sites. An expert pathologist annotated these images by outlining AEH/EC regions. We developed an end-to-end machine learning model with mixed supervision. The model is based on image patches extracted from pathologist-annotated AEH/EC regions. Either an unsupervised deep learning architecture (Autoencoder or ResNet50), or non-deep learning (radiomics feature extraction) is used to embed the images into a low-dimensional space, followed by fully connected layers for binary prediction, which was trained with binary responder/non-responder labels established by pathologists. We used stratified sampling to partition the dataset into a development set and a test set for internal validation of the performance of our models.ResultsThe autoencoder model yielded an AUROC of 0.80 with 95% CI [0.63, 0.95] on the independent test set for the task of predicting a patient with AEH/EC as a responder vs non-responder to hormonal treatment.ConclusionsThese findings demonstrate the potential of using mixed supervised machine learning models on WSIs for predicting the response to hormonal treatment in AEH/EC patients.
Percutaneous nephrostomy (PCN) is a minimally invasive procedure used in kidney surgery. PCN needle placement is of great importance for the following successful renal surgery. In this study, we designed and built an endoscopic polarization-sensitive optical coherence tomography (PS-OCT) system for the PCN needle guidance. Compared to traditional OCT, PS-OCT will allow more accurate differentiation of the renal tissue types in front of the needle. In the experiment, we imaged different renal tissues from human kidneys using the PS-OCT endoscope. Furthermore, deep learning methods were applied for automatic recognition of different tissue types.
Fourier Ptychography Microscopy (FPM) is considered as one emerging technology for the development of high efficiency and low-cost microscopic scanners. One of the major advantages of FPM is its large depth of field (DOF), which significantly reduces the mechanical accuracy requirement of the scanning stages. In this study, we experimentally measured the DOF for our FPM prototype under different illumination conditions. The measurements were based on the theory that the DOF is considered as the range along optical axis for which the contrast is above 80% of the maximum when adjusting the focus location. Accordingly, the contrast is estimated using the bar pattern on the standard resolution target USAF1951 where the modulation transfer function (MTF) curve value drops to 0.5. During the experiment, the FPM prototype is equipped with a 4×/0.13 NA objective lens, and the DOF measurement was conducted with conventional single LED illumination and symmetric illumination. The results demonstrate that the DOF of the single LED illumination FPM is 15.3 µm, which is close to the DOF of the objective lens (14.5 µm). The DOF increases to 22.7 µm when symmetric illumination is adopted, which agrees with the theoretical conclusion. This investigation provides meaningful information for the future optimization of the FPM-based microscopic digitizers.
Guidance of epidural needle is important for the safe and efficient epidural anesthesia procedure. In this study, we built an endoscopic system based on polarization-sensitive optical coherence tomography (PS-OCT). We used pig backbones to test the feasibility of our PS-OCT endoscopic system. Different spinal tissue layers that epidural needle punctures through including subcutaneous fat, ligament, ligamentum flavum, epidural space and spinal cord were imaged and analyzed. They showed different imaging features on the PS-OCT imaging results. Furthermore, we applied deep-learning methods to classify those tissue types automatically to improve the recognition efficiency.
When the epidural needle is punctured into human body during epidural anesthesia surgery, the location of the needle tip is of great importance. In our study, we developed an OCT endoscopic system to help locate the needle tip in real time. Backbones from pigs were utilized to test our system. According to the tissue types that epidural needle punctures through, we imaged five different tissues (fat, ligament, flavum, epidural space and spinal cord). Furthermore, deep-learning methods were used to automatically distinguish the tissue types and predict the distance between the needle tip and the spinal cord. We achieved an average prediction accuracy of 96.65% in tissue classification, and an absolute percentage error at 3.05%±0.55% in distance measurement.
Metaphase chromosome karyotyping plays an important role in the diagnosis of certain cancers and some genetic diseases by detecting chromosome abnormalities. For this technique, high magnification objective lens is used to ensure the chromosome’s band pattern sharpness, but the small field of view (FOV) of the lens makes the imaging of chromosomes very tedious and time consuming. The purpose of this study is to verify the use of the Fourier ptychography microscopy (FPM) system in high-resolution karyotyping. Based on our former study, we further expanded the theoretical NA of the FPM system to 1.11 with a 20×/0.4 NA objective lens and higher illumination angles. To evaluate the resolving power of the FPM system, a 1951 USAF resolution target was imaged to create the modulation transfer function (MTF) curves. The performance of the FPM system was also assessed by imaging chromosomes acquired from blood and bone marrow pathological samples. The results were compared with a conventional 100×/1.45 NA oil immersion objective lens. The MTF curves demonstrate that the contrast of the FPM system is inferior but close to the 100× objective lens (1.45 NA). As compared to the images acquired by the 100×/1.45 NA oil immersion objective lens, the chromosome images recovered by the FPM system contain all the band patterns, despite the loss of some fine details. This study initially verified that the high NA FPM system can guarantee the sharpness of chromosome band patterns as the conventional high magnification oil immersion objective lens, while enabling a large FOV without the utilization of oil immersion medium.
The study aims to develop a novel computer-aided diagnosis (CAD) scheme for mammographic breast mass classification using semi-supervised learning. Although supervised deep learning has achieved huge success across various medical image analysis tasks, its success relies on large amounts of high-quality annotations, which can be challenging to acquire in practice. To overcome this limitation, we propose employing a semi-supervised method, i.e., virtual adversarial training (VAT), to leverage and learn useful information underlying in unlabeled data for better classification of breast masses. Accordingly, our VAT-based models have two types of losses, namely supervised and virtual adversarial losses. The former loss acts as in supervised classification, while the latter loss aims at enhancing the model’s robustness against virtual adversarial perturbation, thus improving model generalizability. To evaluate the performance of our VAT-based CAD scheme, we retrospectively assembled a total of 1024 breast mass images, with equal number of benign and malignant masses. A large CNN and a small CNN were used in this investigation, and both were trained with and without the adversarial loss. When the labeled ratios were 40% and 80%, VAT-based CNNs delivered the highest classification accuracy of 0.740±0.015 and 0.760±0.015, respectively. The experimental results suggest that the VAT-based CAD scheme can effectively utilize meaningful knowledge from unlabeled data to better classify mammographic breast mass images.
Significance: Searching analyzable metaphase chromosomes is a critical step for the diagnosis and treatment of leukemia patients, and the searching efficiency is limited by the difficulty that the conventional microscopic systems have in simultaneously achieving high resolution and a large field of view (FOV). However, this challenge can be addressed by Fourier ptychography microscopy (FPM) technology.
Aim: The purpose of this study is to investigate the feasibility of utilizing FPM to reconstruct high-resolution chromosome images.
Approach: An experimental FPM prototype, which was equipped with 4 × / 0.1 NA or 10 × / 0.25 NA objective lenses to achieve a theoretical equivalent NA of 0.48 and 0.63, respectively, was developed. Under these configurations, we first generated the system modulation transfer function (MTF) curves to assess the resolving power. Next, a group of analyzable metaphase chromosomes were imaged by the FPM system, which were acquired from the peripheral blood samples of the leukemia patients. The chromosome feature qualities were evaluated and compared with the results accomplished by the corresponding conventional microscopes.
Results: The MTF curve results indicate that the resolving power of the 4 × / 0.1 NA FPM system is equivalent and comparable to the 20 × / 0.4 NA conventional microscope, whereas the performance of the 10 × / 0.25 NA FPM system is close to the 60 × / 0.95 NA conventional microscope. When imaging the chromosomes, the feature qualities of the 4 × / 0.1 NA FPM system are comparable to the results under the conventional 20 × / 0.4 NA lens, whereas the feature qualities of the 10 × / 0.25 NA FPM system are better than the conventional 60 × / 0.95 NA lens and comparable to the conventional 100 × / 1.25 NA lens.
Conclusions: This study initially verified that it is feasible to utilize FPM to develop a high-resolution and wide-field chromosome sample scanner.
Automatic classification of epithelium and stroma regions on histopathological images is critically important in digital pathology. Although many studies have been conducted in this research area, few investigations have been focused on model generalizability between different types of tissue samples. The objective of this study is to initially verify the classification effectiveness of a sufficiently optimized transfer model. Accordingly, two datasets were assembled, which contain 157 breast cancer images (Dataset I) and 11 ovarian cancer images (Dataset II), respectively. A computer aided detection (CAD) scheme was developed for this classification task. The scheme first divided each image into small regions of interest (ROI) containing only epithelium or stroma tissues, using multi-resolution super-pixel algorithm. Then, a total of 26 quantitative features were computed for each ROI, which were used as the input of five different machine learning classifiers, namely, linear support vector machine (SVM), linear discriminant analysis (LDA), logistic regression, decision tree and k-nearest neighbors (KNN). The scheme was trained and optimized on Dataset I, and five-fold cross validation strategy was utilized for performance evaluation. After the scheme was sufficiently optimized on Dataset I, it was applied “as is” on dataset II. The results of the breast cancer dataset show that linear SVM achieved the highest classification accuracy of 0.910. When applied on the 11 ovarian cancer cases (Dataset II), the SVM model achieved an average classification accuracy of 0.744. This preliminary study initially demonstrates the model transfer performance for epithelium-stroma classification task.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.