Successful cancer treatment continues to elude modern medicine and its arsenal of therapeutic strategies. Therapy resistance is driven by significant tumor heterogeneity, complex interactions between malignant, microenvironmental and immune cells and cross talk between signaling pathways. Advances in molecular characterization technologies such as next generation sequencing have helped unravel this network of interactions and identify druggable therapeutic targets. Tyrosine kinase inhibitors (TKI) are a class of drugs seeking to inhibit signaling pathways critical to sustaining proliferative signaling, resisting cell death, and the other hallmarks of cancer. While tumors may initially respond to TKI therapy, disease progression is near universal due to mechanisms of acquired resistance largely involving cellular signaling pathway reprogramming. With the ultimate goal of improved TKI therapeutic efficacy our group has developed intracellular paired agent imaging (iPAI) to quantify drug target interactions and oligonucleotide conjugated antibody (Ab-oligo) cyclic immunofluorescence (cycIF) imaging to characterize perturbed signaling pathways in response to therapy. iPAI uses spectrally distinct, fluorescently labeled targeted and untargeted drug derivatives, correcting for non-specific drug distribution and facilitating quantitative assessment of the drug binding before and after therapy. Ab-oligo cycIF exploits in situ hybridization of complementary oligonucleotides for biomarker labeling while oligonucleotide modifications facilitate signal removal for sequential rounds of fluorescent tagging and imaging. Aboligo CycIF is capable of generating extreme multi-parametric images for quantifying total and phosphorylated protein expression to quantify protein activation, expression, and spatial distribution. Together iPAI and Ab-oligo cycIF can be applied to interrogate drug uptake and target binding as well as changes to heterogenous cell populations within tumors that drive variable therapeutic responses in patients.
Purpose: Pathologists rely on relevant clinical information, visual inspection of stained tissue slide morphology, and sophisticated molecular diagnostics to accurately infer the biological origin of secondary metastatic cancer. While highly effective, this process is expensive in terms of time and clinical resources. We seek to develop and evaluate a computer vision system designed to reasonably infer metastatic origin of secondary liver cancer directly from digitized histopathological whole slide images of liver biopsy.
Approach: We illustrate a two-stage deep learning approach to accomplish this task. We first train a model to identify spatially localized regions of cancerous tumor within digitized hematoxylin and eosin (H&E)-stained tissue sections of secondary liver cancer based on a pathologist’s annotation of several whole slide images. Then, a second model is trained to generate predictions of the cancers’ metastatic origin belonging to one of three distinct clinically relevant classes as confirmed by immunohistochemistry.
Results: Our approach achieves a classification accuracy of 90.2% in determining metastatic origin of whole slide images from a held-out test set, which compares favorably to an established clinical benchmark by three board-certified pathologists whose accuracies ranged from 90.2% to 94.1% on the same prediction task.
Conclusions: We illustrate the potential impact of deep learning systems to leverage morphological and structural features of H&E-stained tissue sections to guide pathological and clinical determination of the metastatic origin of secondary liver cancers.
This work applies deep variational autoencoder learning architecture to study multi-cellular growth characteristics of human mammary epithelial cells in response to diverse microenvironment perturbations. Our approach introduces a novel method of visualizing learned feature spaces of trained variational autoencoding models that enables visualization of principal features in two dimensions. We find that unsupervised learned features more closely associate with expert annotation of cell colony organization than biologically-inspired hand-crafted features, demonstrating the utility of deep learning systems to meaningfully characterize features of multi-cellular growth characteristics in a fully unsupervised and data-driven manner.
Successful cancer treatment continues to elude modern medicine and its arsenal of therapeutic strategies. Therapy resistance is driven by significant tumor heterogeneity, complex interactions between malignant, microenvironmental and immune cells and cross talk between signaling pathways. Advances in molecular characterization technologies such as next generation sequencing have helped unravel this network of interactions and have vastly affected how cancer is diagnosed and treated. However, the translation of complex genomic analyses to pathological diagnosis remains challenging using conventional immunofluorescence (IF) staining, which is typically limited to 2-5 antigens. Numerous strategies to increase distinct antigen detection on a single sample have been investigated, but all have deleterious effects on the tissue limiting the maximum number of biomarkers that can be imaged on a single sample and none can be seamlessly integrated into routine clinical workflows. To facilitate ready integration into clinical histopathology, we have developed a novel cyclic IF (cycIF) technology based on antibody conjugated oligonucleotides (Ab-oligos). In situ hybridization of complementary oligonucleotides (oligos) facilitates biomarker labeling for imaging on any conventional fluorescent microscope. We have validated a variety of oligo configurations and their respective signal removal strategies capable of diminishing fluorescent signal to levels of autofluorescence before subsequent staining cycles. Robust signal removal is performed without the employment of harsh conditions or reagents, maintaining tissue integrity and antigenicity for higher dimensionality immunostaining of a single sample. Our platform Ab-oligo cycIF technology uses conventional fluorophores and microscopes, allowing for dissemination to a broad audience and congruent integration into clinical histopathology workflows.
Multiplexed imaging such as multicolor immunofluorescence staining, multiplexed immunohistochemistry (mIHC) or cyclic immunofluorescence (cycIF) enables deep assessment of cellular complexity in situ and, in conjunction with standard histology stains like hematoxylin and eosin (H and E), can help to unravel the complex molecular relationships and spatial interdependencies that undergird disease states. However, these multiplexed imaging methods are costly and can degrade both tissue quality and antigenicity with each successive cycle of staining. In addition, computationally intensive image processing such as image registration across multiple channels is required. We have developed a novel method, speedy histopathological-to-immunofluorescent translation (SHIFT) of whole slide images (WSIs) using conditional generative adversarial networks (cGANs). This approach is rooted in the assumption that specific patterns captured in IF images by stains like DAPI, pan-cytokeratin (panCK), or α-smooth muscle actin (α-SMA) are encoded in H and E images, such that a SHIFT model can learn useful feature representations or architectural patterns in the H and E stain that help generate relevant IF stain patterns. We demonstrate that the proposed method is capable of generating realistic tumor marker IF WSIs conditioned on corresponding H and E-stained WSIs with up to 94.5% accuracy in a matter of seconds. Thus, this method has the potential to not only improve our understanding of the mapping of histological and morphological profiles into protein expression profiles, but also greatly increase the efficiency of diagnostic and prognostic decision-making.
This study has brought together image processing, clustering and spatial pattern analysis to quantitatively analyze hematoxylin and eosin-stained (HE) tissue sections. A mixture of tumor and normal cells (intratumoral heterogeneity) as well as complex tissue architectures of most samples complicate the interpretation of their cytological profiles. To address these challenges, we develop a simple but effective methodology for quantitative analysis for HE section. We adopt comparative analyses of spatial point patterns to characterize spatial distribution of different nuclei types and complement cellular characteristics analysis. We demonstrate that tumor and normal cell regions exhibit significant differences of lymphocytes spatial distribution or lymphocyte infiltration pattern.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.