Artificial Intelligence-Powered Analyzer Reduces Inter-observer Variation in PD-L1 Tumor Proportion Score of Non-Small Cell Lung Cancer
Seokhwi Kim, Sangjoon Choi, Soo Ick Cho, Hyojin Kim, Minuk Ma, Seonwook Park, Sergio Pereira, Brian Jaehong Aum, Seunghwan Shin, Kyunghyun Paeng, Donggeun Yoo, Wonkyung Jung, Chan-Young Ock, Se-Hoon Lee, Jin-Haeng Chung,Yoon-La Choi
USCAP, 2022
Background: Programmed death ligand 1 (PD-L1) expression level is a prognostic marker in predicting response to immune checkpoint inhibitors (ICIs) in advanced non-small cell lung cancer (NSCLC). However, manual evaluation of the PD-L1 tumor proportion score (TPS) often leads to inter-observer variation. Whether artificial intelligence (AI) assistance could improve the accurate TPS reading and hence lead to better prediction of the ICI response have not been studied yet.
Design: An AI-powered analyzer, LUNIT SCOPE PD-L1 TPS was developed with a total of 393,565 tumor cells from 802 whole-slide images (WSIs) of NSCLC, annotated by board-certified pathologists. Three independent pathologists scored PD-L1 TPS in an external cohort of 479 patients. They then had chances to revise the initial TPS groups (<1%, 1%–49%, and ≥50%) with AI assistance. They independently reviewed cases that the TPS group had been changed after AI model assistance despite the initial concordance among them (human misinterpretation) and cases that the group had not been changed despite the AI suggestion of other TPS groups (AI misinterpretation). The effect on clinical outcome of AI-assisted revised TPS group was also analyzed in 430 patients treated with second or later ICIs.
Results: Initially, the three pathologists concordantly scored TPS in 81.4% of cases. Following the revision of their score by adopting AI’s suggestion of TPS group other than their baseline value (N = 91, 93, and 107 for each pathologist), the overall concordance rate was increased to 90.2% (P < 0.001). The potential human misinterpretation cases include simple under-/overestimation (74.2%), difficulty due to artifacts (12.9%), and difficulty due to cellular morphology (12.9%) (Figure 1). The potential AI misinterpretation cases include false-negative readings of PD-L1-negative normal epithelium (66.7%), false-negative readings of PD-L1-positive tumor cell (19.0%), and false-positive readings of PD-L1-positive macrophage (14.3%). The revised TPS group with AI assistance predicted objective response rate, progression-free survival, and overall survival better than the baseline group without AI assistance (Figure 2).
Conclusions: Assistance with the AI-powered TPS analyzer improved the pathologists’ consensus and prediction of the clinical response. In utilizing AI models, pathologists could interpret PD-L1 expression more accurately by understanding the weaknesses of pathologists and AI models.