Back to List

Deep learning from HE slides predicts the clinical benefit from adjuvant chemotherapy in hormone receptor-positive breast cancer patients

Soo Youn Cho et al. — Scientific Reports (2021)


We hypothesized that a deep-learning algorithm using HE images might be capable of predicting the benefits of adjuvant chemotherapy in cancer patients. HE slides were retrospectively collected from 1343 de-identified breast cancer patients at the Samsung Medical Center and used to develop the Lunit SCOPE algorithm. Lunit SCOPE was trained to predict the recurrence using the 21-gene assay (Oncotype DX) and histological parameters. The risk prediction model predicted the Oncotype DX score > 25 and the recurrence survival of the prognosis validation cohort and TCGA cohorts. The most important predictive variable was the mitotic cells in the cancer epithelium. Of the 363 patients who did not receive adjuvant therapy, 104 predicted high risk had a significantly lower survival rate. The top-300 genes highly correlated with the predicted risk were enriched for cell cycle, nuclear division, and cell division. From the Oncotype DX genes, the predicted risk was positively correlated with proliferation-associated genes and negatively correlated with prognostic genes from the estrogen category. An integrative analysis using Lunit SCOPE predicted the risk of cancer recurrence and the early-stage hormone receptor-positive breast cancer patients who would benefit from adjuvant chemotherapy.


Breast cancer is the most common cancer in women worldwide, and hormone-receptor (HR)-positive, lymph node-negative diseases account for nearly half of all breast cancer cases1,2. As excellent prognosis in many of these patients have been known, many efforts to identify those patients with high risk of recurrence, who would benefit from adjuvant chemotherapy (ACTx), were made using gene expression profiling3,4,5,6. Currently, several multigene assays, such as the 21-gene assay (Oncotype DX), PAM50, and Mammaprint, are used to stratify patients and guide ACTx according to the recurrence risk in HR-positive, and lymph node- negative breast cancer after extensive clinical validation7,8.

Despite the proven clinical utility of RS for the 21-gene assay, its effectiveness in patients with HR-positive, lymph node-negative, early stage breast cancer remains controversial, along with its financial burden in countries outside of the US9,10. Moreover, the instability of RNA extracted from formalin-fixed paraffin-embedded (FFPE) tissue in real-world practice might compromise its accuracy and interfere with the appropriate translation of the RS results11. Therefore, the development of a simpler and more efficient method for assessing recurrence risk using permanent tissue is necessary. As the RS from the 21-gene assay is mainly characterized by the proliferation genes group score (MKI67STK15BIRC5, CCNB1, and MYBL2) and the mitotic count is associated with the RS7, a comprehensive pathological examination of mitosis and other cell–cell interactions features, consistently reflects the RS.

Thus, we developed a deep learning (DL)-based HE image analyzer called Lunit SCOPE that identifies and quantifies various histological parameters from HE-stained whole slide images (WSIs). Previously, the Lunit SCOPE was shown to accurately detect tumor cells as well as other cells in a microenvironment, and it clearly predicted mitosis in each cell in breast cancer12. Based on The Cancer Genome Atlas (TCGA) pan-cancer analysis, Lunit SCOPE was able to predict an abundance of cancer-associated stroma in pancreatic adenocarcinoma and a consensus of molecular subtype 4 of colon cancer13, as well as tumor-infiltrating lymphocytes in immunogenic tumors such as renal cell carcinoma, melanoma, and urothelial cancer14.

As Lunit SCOPE accurately identifies the comprehensive features of HE slides, especially regarding mitotic count and the infiltration of immune cells or stromal cells, we hypothesized that histological parameters analyzed using Lunit SCOPE would predict the RS from the 21-gene assay, revealing potential prognostic and predictive biomarkers of ACTx in early stage hormone receptor-positive breast cancer.


Detection of various cell types in the breast cancer HE slides

The Lunit SCOPE divides the HE slide image into histological parameters through three panels, including the tissue, structure, and cell panel. The process used to develop the Lunit SCOPE and workflow of this study are illustrated in Fig. 1 (detailed description in the Supplementary Methods). Each panel is an independent multi-class prediction model trained using curated ground-truth annotations from expert pathologists. The panels decipher the histological parameters in the image divided into small patch images and ultimately return the aggregated count values corresponding to the tissue, structure, and cell from the WSIs. The performance of the three panels is described in Supplementary Table 1.

Read the full paper

Soo Youn Cho, Jeong Hoon Lee, Jai Min Ryu, Jeong Eon Lee, Eun Yoon Cho, Chang Ho Ahn, Kyunghyun Paeng, Inwan Yoo, Chan-Young Ock & Sang Yong Song

Scientific Reports (2021)

Read more