We hypothesized that a deep-learning algorithm using HE images might be capable of predicting the benefits of adjuvant chemotherapy in cancer patients. HE slides were retrospectively collected from 1343 de-identified breast cancer patients at the Samsung Medical Center and used to develop the Lunit SCOPE algorithm. Lunit SCOPE was trained to predict the recurrence using the 21-gene assay (Oncotype DX) and histological parameters. The risk prediction model predicted the Oncotype DX score > 25 and the recurrence survival of the prognosis validation cohort and TCGA cohorts. The most important predictive variable was the mitotic cells in the cancer epithelium. Of the 363 patients who did not receive adjuvant therapy, 104 predicted high risk had a significantly lower survival rate. The top-300 genes highly correlated with the predicted risk were enriched for cell cycle, nuclear division, and cell division. From the Oncotype DX genes, the predicted risk was positively correlated with proliferation-associated genes and negatively correlated with prognostic genes from the estrogen category. An integrative analysis using Lunit SCOPE predicted the risk of cancer recurrence and the early-stage hormone receptor-positive breast cancer patients who would benefit from adjuvant chemotherapy.
Breast cancer is the most common cancer in women worldwide, and hormone-receptor (HR)-positive, lymph node-negative diseases account for nearly half of all breast cancer cases1,2. As excellent prognosis in many of these patients have been known, many efforts to identify those patients with high risk of recurrence, who would benefit from adjuvant chemotherapy (ACTx), were made using gene expression profiling3,4,5,6. Currently, several multigene assays, such as the 21-gene assay (Oncotype DX), PAM50, and Mammaprint, are used to stratify patients and guide ACTx according to the recurrence risk in HR-positive, and lymph node- negative breast cancer after extensive clinical validation7,8.
Despite the proven clinical utility of RS for the 21-gene assay, its effectiveness in patients with HR-positive, lymph node-negative, early stage breast cancer remains controversial, along with its financial burden in countries outside of the US9,10. Moreover, the instability of RNA extracted from formalin-fixed paraffin-embedded (FFPE) tissue in real-world practice might compromise its accuracy and interfere with the appropriate translation of the RS results11. Therefore, the development of a simpler and more efficient method for assessing recurrence risk using permanent tissue is necessary. As the RS from the 21-gene assay is mainly characterized by the proliferation genes group score (MKI67, STK15, BIRC5, CCNB1, and MYBL2) and the mitotic count is associated with the RS7, a comprehensive pathological examination of mitosis and other cell–cell interactions features, consistently reflects the RS.
Thus, we developed a deep learning (DL)-based HE image analyzer called Lunit SCOPE that identifies and quantifies various histological parameters from HE-stained whole slide images (WSIs). Previously, the Lunit SCOPE was shown to accurately detect tumor cells as well as other cells in a microenvironment, and it clearly predicted mitosis in each cell in breast cancer12. Based on The Cancer Genome Atlas (TCGA) pan-cancer analysis, Lunit SCOPE was able to predict an abundance of cancer-associated stroma in pancreatic adenocarcinoma and a consensus of molecular subtype 4 of colon cancer13, as well as tumor-infiltrating lymphocytes in immunogenic tumors such as renal cell carcinoma, melanoma, and urothelial cancer14.
As Lunit SCOPE accurately identifies the comprehensive features of HE slides, especially regarding mitotic count and the infiltration of immune cells or stromal cells, we hypothesized that histological parameters analyzed using Lunit SCOPE would predict the RS from the 21-gene assay, revealing potential prognostic and predictive biomarkers of ACTx in early stage hormone receptor-positive breast cancer.
The Lunit SCOPE divides the HE slide image into histological parameters through three panels, including the tissue, structure, and cell panel. The process used to develop the Lunit SCOPE and workflow of this study are illustrated in Fig. 1 (detailed description in the Supplementary Methods). Each panel is an independent multi-class prediction model trained using curated ground-truth annotations from expert pathologists. The panels decipher the histological parameters in the image divided into small patch images and ultimately return the aggregated count values corresponding to the tissue, structure, and cell from the WSIs. The performance of the three panels is described in Supplementary Table 1.
Soo Youn Cho, Jeong Hoon Lee, Jai Min Ryu, Jeong Eon Lee, Eun Yoon Cho, Chang Ho Ahn, Kyunghyun Paeng, Inwan Yoo, Chan-Young Ock & Sang Yong Song
Abstract: Deep learning-based predictive biomarker for adjuvant chemotherapy in early-stage hormone receptor-positive breast cancer
Abstract: Pan-cancer analysis of tumor microenvironment using deep learning-based cancer stroma and immune profiling in H&E images
Abstract: Deep learning-based predictive biomarker for immune checkpoint inhibitor response in metastatic non-small cell lung cancer
Abstract: Comprehensive deep learning analysis of H&E tissue phenomics reveals distinct immune landscape and transcriptomic enrichment profile among immune inflamed, excluded and desert subtypes...
Abstract: Deep-learning analysis of H&E images to define three immune phenotypes to reveal loss-of-target in excluded immune cells as a novel resistance mechanism of immune checkpoint inhibitor...
Abstract: Deep learning-based immune phenotype analysis reveals distinct resistance pattern of immune checkpoint inhibitor in non-small cell lung cancer
Reducing Domain Gap by Reducing Style Bias
Predicting breast tumor proliferation from whole-slide images: The TUPAC16 challenge
Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses
Abstract: Artificial intelligence-powered spatial analysis of tumor infiltrating lymphocytes (TIL) to reflect target gene expressions of novel immuno-oncology agents.
Abstract: Distinct subset of immune cells assessed by multiplex immunohistochemistry correlates with immune phenotype classified by ...
Abstract : Pathologic validation of artificial intelligence-powered prediction of combined positive score of PD-L1 immunohistochemistry in urothelial carcinoma.
Abstract : Clinical performance of artificial intelligence-powered annotation of tumor cell PD-L1 expression for treatment of immune-checkpoint inhibitor (ICI) ...
Abstract : Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes predicts survival after immune checkpoint inhibitor therapy across multiple cancer types.
Abstract : Deep learning based radiomic biomarker for predicting the presence of high-grade histologic patterns in lung adenocarcinoma
Abstract : Artificial intelligence-powered tissue analysis reveals distinct tumor-infiltrating lymphocyte profile as a potential biomarker of molecular subtypes in endometrial cancer
Abstract : Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes reveals distinct genomic profile of immune excluded phenotype in pan-carcinoma
Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems
Abstract : Interim results of phase I dose escalation study of YBL-006: A novel anti-PD-1 monoclonal antibody in advanced solid tumors
Abstract : Assistance with an artificial intelligence-powered PD-L1 analyzer reduces interobserver variation in pathologic reading of tumor proportion score in non-small cell lung cancer
Abstract : AI-powered whole-slide image analysis of tumor-infiltrating lymphocytes for prediction of prognosis in colorectal cancer
Deep Learning Analysis of CT Images Reveals High-Grade Pathological Features to Predict Survival in Lung Adenocarcinoma
Artificial Intelligence–Powered Spatial Analysis of Tumor-Infiltrating Lymphocytes as Complementary Biomarker for Immune Checkpoint Inhibition in Non–Small-Cell Lung Cancer
Artificial Intelligence–Powered Hematoxylin and Eosin Analyzer Reveals Distinct Immunologic and Mutational Profiles among Immune Phenotypes in Non–Small-Cell Lung Cancer
Abstract : Artificial Intelligence-Powered Analyzer Reduces Inter-observer Variation in PD-L1 Tumor Proportion Score of Non-Small Cell Lung Cancer
Abstract : Artificial Intelligence-Powered Tumor Purity Assessment From H&E Whole Slide Images Correlates...
Abstract : Deep learning-based H&E analyzer reveals distinct immune profiles and clinical outcomes among immune phenotypes in uterine corpus endometrial carcinoma
Abstract : Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes reveals immune-excluded phenotype is correlated with TGF-beta pathway related genomic aberrations
Observer Performance Study to Examine the Feasibility of the AI-powered PD-L1 Analyzer to Assist Pathologists’ Assessment of PD-L1 Expression Using Tumor Proportion Score in Non-Small Cell Lung Cancer
Artificial intelligence-powered human epidermal growth factor receptor 2 (HER2) analyzer in breast cancer as an assistance tool for pathologists to reduce interobserver variation
Artificial intelligence-powered whole-slide image analyzer reveals a distinctive distribution of tumor-infiltrating lymphocytes in neuroendocrine tumors and carcinomas
Artificial Intelligence (AI) - powered spatial analysis of tumor-infiltrating lymphocytes (TIL) for prediction of response to neoadjuvant chemotherapy (NAC) in triple negative breast cancer (TNBC)
Artificial intelligence (AI)-powered pathology image analysis merged with spatial transcriptomics reveals distinct TIGIT expression in the immune-excluded tumor-infiltrating lymphocytes
Trastuzumab plus FOLFOX for Gemcitabine/Cisplatin refractory HER2-positive biliary tract cancer: a multi-institutional phase II trial of the Korean Cancer Study Group (KCSG-HB19-14)
The Inflamed Immune Phenotype (IIP): a clinically actionable artificial intelligence (AI)-based biomarker predictive of immune checkpoint inhibitor (ICI) outcomes across >16 primary tumor types
Safety and efficacy of YBL-006, an anti-PD-1 monoclonal antibody in advanced solid tumors: a phase I study
Robust artificial intelligence-powered imaging biomarker based on mammography for risk prediction of breast cancer.