{"id":1594,"date":"2020-07-14T20:11:33","date_gmt":"2020-07-14T20:11:33","guid":{"rendered":"https:\/\/www.lunit.io\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/"},"modified":"2025-11-02T02:55:11","modified_gmt":"2025-11-02T02:55:11","slug":"automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration","status":"publish","type":"publication","link":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/","title":{"rendered":"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration"},"content":{"rendered":"

Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration<\/h3>\n

Eui Jin Hwang, Hyungjin Kim, Jong Hyuk Lee, et al.<\/p>\n

European Radiology, 2020<\/strong><\/p>\n

Abstract<\/strong>
\nObjectives<\/strong>
\nTo evaluate the calibration of a deep learning (DL) model in a diagnostic cohort and to improve model\u2019s calibration through recalibration procedures.<\/p>\n

Methods<\/strong>
\nChest radiographs (CRs) from 1135 consecutive patients (M:F\u2009=\u2009582:553; mean age, 52.6 years) who visited our emergency department were included. A commercialized DL model was utilized to identify abnormal CRs, with a continuous probability score for each CR. After evaluation of the model calibration, eight different methods were used to recalibrate the original model based on the probability score. The original model outputs were recalibrated using 681 randomly sampled CRs and validated using the remaining 454 CRs. The Brier score for overall performance, average and maximum calibration error, absolute Spiegelhalter\u2019s Z for calibration, and area under the receiver operating characteristic curve (AUROC) for discrimination were evaluated in 1000-times repeated, randomly split datasets.<\/p>\n

Results<\/strong>
\nThe original model tended to overestimate the likelihood for the presence of abnormalities, exhibiting average and maximum calibration error of 0.069 and 0.179, respectively; an absolute Spiegelhalter\u2019s Z value of 2.349; and an AUROC of 0.949. After recalibration, significant improvements in the average (range, 0.015\u20130.036) and maximum (range, 0.057\u20130.172) calibration errors were observed in eight and five methods, respectively. Significant improvement in absolute Spiegelhalter\u2019s Z (range, 0.809\u20134.439) was observed in only one method (the recalibration constant). Discriminations were preserved in six methods (AUROC, 0.909\u20130.949).<\/p>\n

Conclusion<\/strong>
\nThe calibration of DL algorithm can be augmented through simple recalibration procedures. Improved calibration may enhance the interpretability and credibility of the model for users.<\/p>\n

Read the full paper<\/strong><\/a><\/p>\n","protected":false},"featured_media":0,"template":"","publication-oncology":[],"publication-region":[87],"publication-type":[],"radiology":[104,106,96],"class_list":["post-1594","publication","type-publication","status-publish","hentry","publication-region-asia","radiology-chest","radiology-improving-accuracy","radiology-lunit-insight"],"acf":[],"yoast_head":"\nAutomated identification of chest radiographs with referable abnormality with deep learning: need for recalibration - Lunit<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration - Lunit\" \/>\n<meta property=\"og:description\" content=\"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration Eui Jin Hwang, Hyungjin Kim, Jong Hyuk Lee, et al. European Radiology, 2020 Abstract Objectives To evaluate the calibration of a deep learning (DL) model in a diagnostic cohort and to improve model\u2019s calibration through recalibration procedures. Methods Chest radiographs (CRs) […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/\" \/>\n<meta property=\"og:site_name\" content=\"Lunit\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-02T02:55:11+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@lunit_ai\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/publication\\\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\\\/\",\"url\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/publication\\\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\\\/\",\"name\":\"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration - Lunit\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/#website\"},\"datePublished\":\"2020-07-14T20:11:33+00:00\",\"dateModified\":\"2025-11-02T02:55:11+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/publication\\\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.lunit.io\\\/en\\\/publication\\\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/publication\\\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/#website\",\"url\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/\",\"name\":\"Lunit\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/#organization\",\"name\":\"Lunit\",\"url\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/Logo-black.svg\",\"contentUrl\":\"https:\\\/\\\/www.lunit.io\\\/en\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/Logo-black.svg\",\"width\":189,\"height\":52,\"caption\":\"Lunit\"},\"image\":{\"@id\":\"https:\\\/\\\/www.lunit.io\\\/ko\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/lunit_ai\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/lunit-inc\",\"https:\\\/\\\/x.com\\\/lunitoncology\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration - Lunit","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/","og_locale":"en_US","og_type":"article","og_title":"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration - Lunit","og_description":"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration Eui Jin Hwang, Hyungjin Kim, Jong Hyuk Lee, et al. European Radiology, 2020 Abstract Objectives To evaluate the calibration of a deep learning (DL) model in a diagnostic cohort and to improve model\u2019s calibration through recalibration procedures. Methods Chest radiographs (CRs) […]","og_url":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/","og_site_name":"Lunit","article_modified_time":"2025-11-02T02:55:11+00:00","twitter_card":"summary_large_image","twitter_site":"@lunit_ai","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/","url":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/","name":"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration - Lunit","isPartOf":{"@id":"https:\/\/www.lunit.io\/ko\/#website"},"datePublished":"2020-07-14T20:11:33+00:00","dateModified":"2025-11-02T02:55:11+00:00","breadcrumb":{"@id":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.lunit.io\/en\/publication\/automated-identification-of-chest-radiographs-with-referable-abnormality-with-deep-learning-need-for-recalibration\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.lunit.io\/en\/"},{"@type":"ListItem","position":2,"name":"Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration"}]},{"@type":"WebSite","@id":"https:\/\/www.lunit.io\/ko\/#website","url":"https:\/\/www.lunit.io\/ko\/","name":"Lunit","description":"","publisher":{"@id":"https:\/\/www.lunit.io\/ko\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.lunit.io\/ko\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.lunit.io\/ko\/#organization","name":"Lunit","url":"https:\/\/www.lunit.io\/ko\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.lunit.io\/ko\/#\/schema\/logo\/image\/","url":"https:\/\/www.lunit.io\/en\/wp-content\/uploads\/2025\/10\/Logo-black.svg","contentUrl":"https:\/\/www.lunit.io\/en\/wp-content\/uploads\/2025\/10\/Logo-black.svg","width":189,"height":52,"caption":"Lunit"},"image":{"@id":"https:\/\/www.lunit.io\/ko\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/lunit_ai","https:\/\/www.linkedin.com\/company\/lunit-inc","https:\/\/x.com\/lunitoncology"]}]}},"_links":{"self":[{"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/publication\/1594","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/types\/publication"}],"wp:attachment":[{"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/media?parent=1594"}],"wp:term":[{"taxonomy":"publication-oncology","embeddable":true,"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/publication-oncology?post=1594"},{"taxonomy":"publication-region","embeddable":true,"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/publication-region?post=1594"},{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/publication-type?post=1594"},{"taxonomy":"radiology","embeddable":true,"href":"https:\/\/www.lunit.io\/en\/wp-json\/wp\/v2\/radiology?post=1594"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}