Title: Artificial Intelligence of COVID-19 Imaging: A Hammer in Search of a Nail
Abstract: HomeRadiologyVol. 298, No. 3 PreviousNext Reviews and CommentaryFree AccessEditorialArtificial Intelligence of COVID-19 Imaging: A Hammer in Search of a NailRonald M. Summers Ronald M. Summers Author AffiliationsFrom the Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Department of Radiology and Imaging Sciences, National Institutes of Health Clinical Center, 10 Center Dr, Bldg 10, Room 1C224D, MSC 1182, Bethesda, MD 20892-1182.Address correspondence to the author (e-mail: [email protected]).Ronald M. Summers Published Online:Dec 22 2020https://doi.org/10.1148/radiol.2020204226MoreSectionsPDF ToolsAdd to favoritesCiteTrack Citations ShareShare onFacebookXLinked In The coronavirus disease 2019 (COVID-19) pandemic has irrevocably altered our personal and professional lives. More than 1 million people globally have died of the virus. New spikes in infections are occurring worldwide as I write this editorial. The economic impact has been devastating.Early in the pandemic, expectations were raised that chest CT and radiography might play a crucial role in first-line diagnosis of COVID-19 (1). Over time, the reverse transcription polymerase chain reaction test became more sensitive, and clinicians' understanding of the disease and how to treat it improved. Chest CT and radiography has moved to a secondary role. As of this writing, chest CT is not recommended as a first-line test by the American College of Radiology (2).The pandemic reached the United States in January 2020. By March 2020, manuscripts using artificial intelligence (AI) for evaluation of COVID-19 on chest radiographs and CT scans began appearing on preprint servers such as arXiv (3). Within a month, there were over a dozen such manuscripts. Simultaneously, major journals including Radiology began publishing AI articles on COVID-19 (4,5). Most found that these AI systems had high sensitivity for detection of lung opacities due to COVID-19. Subsequent works showed that AI could distinguish COVID-19 from other types of pneumonia (6).The pace of publication of AI articles on COVID-19 is increasing. As of this writing, there are more than 500 such manuscripts on arXiv (Google Scholar search: COVID-19 ["CT" OR "X-ray"] ["machine learning" OR "deep learning" OR "artificial intelligence"] site:arxiv.org) and more than 200 articles on PubMed (COVID-19 ["CT" OR "X-ray"] ["machine learning" OR "deep learning" OR "artificial intelligence"]). The articles are appearing not only in radiology clinical journals but also in technical and general interest scientific journals (7,8). There is a clear appetite for such research despite its repetitiveness and unclear path to clinical utility.Similar enthusiasm for applying AI to pneumonia occurred before COVID-19. During the H1N1 pandemic, an AI system used support vector machine classifiers and texture analysis to detect lung opacities at chest CT (9). The Radiological Society of North America (RSNA) pneumonia detection challenge in 2018 led to more than 1000 teams competing to submit the most effective AI systems for pneumonia detection on chest radiographs (10). But few of those same systems were subsequently applied to COVID-19 (as determined by the paucity of citations to Shih et al [11] on arXiv and PubMed). It is surprising that those systems were not repurposed.To a large extent, the large quantity and rapid publication of articles on AI for COVID-19 are emblematic of current trends in other areas of radiology AI. It is now so much easier to design and conduct a radiology AI experiment. The only prerequisite seems to be possession of a large data set. The AI software tools are available online for free. There are abundant tutorials and recipes for AI research on images in general and radiology images in particular (12).Even the data needed for training and testing AI systems are often not limiting factors. Free online COVID-19 radiology data sets are proliferating. These online data sets include the Cancer Imaging Archive, the British National COVID-19 Chest Imaging Database, and the Valencian Region Medical ImageBank (13–16). The RSNA has developed the Medical Imaging and Data Resource Center, or MIDRC, and RSNA International COVID-19 Open Radiology Database resources and attained buy-in from radiology departments to contribute images from patients with COVID-19 (17,18). At the time of this writing, MIDRC data are not yet online.Public data sets vary in quality and utility. For example, some of the data sets include images but no ancillary data such as annotations, demographics, laboratory results, or outcome data. Some of the annotations are basic such as COVID-19 or no COVID-19. Others are more comprehensive, including anatomic labeling or results of diagnostic antibody tests. Unfortunately, many comprehensive data sets are private and not available to the public. Some AI code and models needed for external use are posted online (19). But there is little evidence to date from investigators independent of the original research teams whether published AI systems generalize to their patients' data.The three major classes of COVID-19 AI research include binary diagnosis (COVID-19 present or absent) (4), segmentation and quantification of the abnormal lung opacities (20), and distinguishing COVID-19 from non-COVID-19 pneumonias (6). Binary diagnosis was one of the first applications studied in depth. More limited areas of investigation include prediction of future need for oxygen therapy or intubation (21), prediction of acute respiratory distress syndrome development (22), generalizability to multinational patient populations (8), integration of imaging and clinical information (23), analysis of serial imaging (24), tailoring steroid treatment (25), and mortality prediction (26). By using natural language processing of clinical reports, opacities on radiology images were included in machine learning models to predict need for intensive care unit admission (27).A recent illustrative example of a binary diagnostic task is the study by Zhang et al (28). The authors studied chest radiographs from 2060 patients with COVID-19 pneumonia and 3148 patients with non-COVID-19 pneumonia. On the test set, their AI system had an area under the receiver operating characteristic curve (AUC) of 0.92 and sensitivities and specificities of 88% and 79%, or 78% and 89% for high sensitivity or high specificity operating thresholds. On a subset of 500 chest radiographs, their AI system achieved an AUC of 0.94 compared with an AUC of 0.85 for three experienced thoracic radiologists. This AUC is typical for these binary diagnostic tasks.An illustrative mortality prediction study is that by Mushtaq et al (26). This was a single institution study of 697 adults with COVID-19 infection confirmed by reverse transcription polymerase chain reaction who presented to the emergency department. A commercial AI system analyzed patients' initial chest radiographs and outputted a score indicating percentage of lung involvement. The score was predictive of mortality (hazard ratio, 2.60) and critical COVID-19 (admission to the intensive care unit or deaths occurring before intensive care unit admission; hazard ratio, 3.40).The basic technical approach of COVID-19 AI research is similar. The first step is to collect a sufficiently large data set. The goalposts determining what is sufficiently large keeps shifting to larger numbers of scans. Typical data sets range from hundreds to thousands of patients' scans. Whether larger data sets are necessary has received inadequate scrutiny. Some published studies include non-COVID scans for training because they are abundant. The data need accurate labels. For some published studies, the labels are binary—the study is positive or negative for COVID-19. For other studies, a segmentation label is manually drawn to identify the extent of the lung abnormality. Some studies use rectangular bounding boxes rather than the more labor-intensive free-form segmentations. The next step is to divide the scans into separate training, validation, and test sets. The machine learning software is taught by using the labeled training data, periodically run on the validation set for fine tuning, and then run only once on the test set. It is ideal to also have an external test data set of patients from a different demographic or institution than the one used for training. High performance on the external test data set increases confidence that the AI is generalizable to new patient populations.There are many choices for the particular deep learning architecture. Some articles pitted different architectures against one another to determine which was most accurate. One such work showed little difference in performance by the different architectures (29). Example deep learning architectures proven successful for detecting COVID-19 include EfficientNet (6), U-Net (30), ResNet (31), and Inf-Net (32). Some studies preprocess the images to segment the lungs before analysis for pulmonary parenchymal opacities (5). U-Net is a very popular deep learning architecture for lung segmentation (33).How does one put this deluge of articles into context? It seems unlikely that an AI system would detect many patients with COVID-19 who had a negative reverse transcription polymerase chain reaction test. Anecdotes will occur. But from a general perspective, this is unlikely to propel dissemination of the AI technology. What about distinguishing COVID-19 from other viral pneumonias? It seems unlikely that clinical decision making would depend on the recommendations of AI, given more definitive laboratory tests are available. Could AI lead to a fully automated interpretation? This has not been the focus of COVID-19 imaging AI to date. Multitask approaches that identify multiple abnormalities at chest imaging besides opacities will be needed, such as universal lesion detection (34,35). What about mortality prediction? Hazard ratios on the order of 2 to 3, as found in the article by Mushtaq et al, are generally insufficient for clinical decision making. While it is possible that prediction of an adverse outcome could lead to more aggressive treatment, it could also lead to unnecessary costs and adverse effects.We are beginning to understand the many risk factors for severe COVID-19 infection and death. These include the presence of underlying conditions such as respiratory or cardiovascular disease, hypertension or diabetes, advanced age, and male sex (36). Thus, there are many opportunities for AI systems that assess or incorporate information about these diseases or patient demographics.We are also beginning to understand some of the nonpulmonary manifestations of COVID-19 (37). These include hepatic and renal injury, neurologic illnesses, and a coagulopathy leading to thrombi. The thrombotic complications can occur anywhere in the body including the mesentery leading to bowel ischemia (38). These nonpulmonary findings are suitable and desirable targets for AI systems.What are the current needs of AI systems for COVID-19 and CT and chest radiography? Public challenges or competitions pitting different AI systems against one another would enable "apples-to-apples" comparisons of performance. More observer performance experiments are necessary to determine whether AI improves clinical interpretation according to reader experience level and reading paradigm (first, concurrent, or second reader). Prospective outcome studies are necessary to determine whether the use of AI leads to changes in patient care, shortened hospitalizations, and reduced morbidity and mortality. Nonradiology clinical information will need to be routinely incorporated into AI models. Assessment of risk and progression of the chronic sequela of COVID-19 infection is necessary. A prospective randomized controlled trial would be exemplary.It is time to move beyond studies showing that AI can detect opacities at CT or chest radiography—this is now well established. Instead, there is a great need for AI systems, based on a combination of imaging, laboratory, and clinical information, that provide actionable predictions otherwise unavailable or less accurate without AI.Disclosures of Conflicts of Interest: R.M.S. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: receives royalties from iCAD, ScanMed, Philips, PingAn, and Translation Holdings. Other relationships: has cooperative research and development agreement with PingAn; received GPU card donations from NVIDIA.Supported by the Intramural Research Program of the National Institutes of Health Clinical Center.References1. Fang Y, Zhang H, Xie J, et al. Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR. Radiology 2020;296(2):E115–E117. Link, Google Scholar2. ACR Recommendations for the use of Chest Radiography and Computed Tomography (CT) for Suspected COVID-19 Infection. https://www.acr.org/Advocacy-and-Economics/ACR-Position-Statements/Recommendations-for-Chest-Radiography-and-CT-for-Suspected-COVID19-Infection. Published 2020. Updated March 22, 2020. Accessed November 1, 2020. Google Scholar3. Yang X, He X, Zhao J, Zhang Y, Zhang S, Xie P. COVID-CT-Dataset: A CT Scan Dataset about COVID-19. ArXiv e-prints 2020:arXiv:2003.13865. [preprint] https://arxiv.org/abs/2003.13865. Posted March 30, 2020. Accessed November 1, 2020. Google Scholar4. Li L, Qin L, Xu Z, et al. Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy. Radiology 2020;296(2):E65–E71. Link, Google Scholar5. Murphy K, Smits H, Knoops AJG, et al. COVID-19 on Chest Radiographs: A Multireader Evaluation of an Artificial Intelligence System. Radiology 2020;296(3):E166–E172. Link, Google Scholar6. Bai HX, Wang R, Xiong Z, et al. Artificial Intelligence Augmentation of Radiologist Performance in Distinguishing COVID-19 from Pneumonia of Other Origin at Chest CT. Radiology 2020;296(3):E156–E165. Link, Google Scholar7. Zhou L, Li Z, Zhou J, et al. A Rapid, Accurate and Machine-Agnostic Segmentation and Quantification Method for CT-Based COVID-19 Diagnosis. IEEE Trans Med Imaging 2020;39(8):2638–2652. Crossref, Medline, Google Scholar8. Harmon SA, Sanford TH, Xu S, et al. Artificial intelligence for the detection of COVID-19 pneumonia on chest CT using multinational datasets. Nat Commun 2020;11(1):4080. Crossref, Medline, Google Scholar9. Yao J, Dwyer A, Summers RM, Mollura DJ. Computer-aided diagnosis of pulmonary infections using texture analysis and support vector machine classification. Acad Radiol 2011;18(3):306–314. Crossref, Medline, Google Scholar10. RSNA Pneumonia Detection Challenge (2018). https://www.rsna.org/en/education/ai-resources-and-training/ai-image-challenge/RSNA-Pneumonia-Detection-Challenge-2018. Published 2018. Accessed November 1, 2020. Google Scholar11. Shih G, Wu CC, Halabi SS, et al. Augmenting the National Institutes of Health Chest Radiograph Dataset with Expert Annotations of Possible Pneumonia. Radiol Artif Intell 2019;1(1):e180041. Link, Google Scholar12. Erickson BJ. Magician's Corner: How to Start Learning about Deep Learning. Radiol Artif Intell 2019;1(4):e190072. Link, Google Scholar13. Jacob J, Alexander D, Baillie JK, et al. Using imaging to combat a pandemic: rationale for developing the UK National COVID-19 Chest Imaging Database. Eur Respir J 2020;56(2):2001809. Crossref, Medline, Google Scholar14. Chest Imaging with Clinical and Genomic Correlates Representing a Rural COVID-19 Positive Population (COVID-19-AR). https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=70226443#70226443171ba531fc374829b21d3647e95f532c. Published 2020. Updated September 15, 2020. Accessed November 1, 2020. Google Scholar15. CT Images in COVID-19. https://wiki.cancerimagingarchive.net/display/Public/CT+Images+in+COVID-19. Published 2020. Updated September 15, 2020. Accessed November 1, 2020. Google Scholar16. Vayá MdlI, Saborit JM, Montell JA, et al. BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients. arXiv preprint arXiv:2006.01174 [preprint]. https://arxiv.org/abs/2006.01174. Posted June 1, 2020. Accessed November 1, 2020. Google Scholar17. RSNA To Collaborate on Open-Source COVID-19 Medical Image Database. https://www.rsna.org/news/2020/july/covid-19-midrc. Published 2020. Updated August 5, 2020. Accessed November 1, 2020. Google Scholar18. Langlotz CP. My Turn: Will Artificial Intelligence Play a Role in Imaging of COVID-19? https://www.rsna.org/news/2020/September/Langlotz-My-Turn-COVID-19-And-AI. Published 2020. Updated September 22, 2020. Accessed November 1, 2020. Google Scholar19. COVID-CT. https://github.com/UCSD-AI4H/COVID-CT. Published 2020. Updated May 11, 2020. Accessed November 1, 2020. Google Scholar20. Li Z, Zhong Z, Li Y, et al. From community-acquired pneumonia to COVID-19: a deep learning-based method for quantitative analysis of COVID-19 on thick-section CT scans. Eur Radiol 2020;30(12):6828–6837. Crossref, Medline, Google Scholar21. Lanza E, Muglia R, Bolengo I, et al. Quantitative chest CT analysis in COVID-19 to predict the need for oxygenation support and intubation. Eur Radiol 2020;30(12):6770–6778. Crossref, Medline, Google Scholar22. Wang Y, Chen Y, Wei Y, et al. Quantitative analysis of chest CT imaging findings with the risk of ARDS in COVID-19 patients: a preliminary study. Ann Transl Med 2020;8(9):594. Crossref, Medline, Google Scholar23. Mei X, Lee HC, Diao KY, et al. Artificial intelligence-enabled rapid diagnosis of patients with COVID-19. Nat Med 2020;26(8):1224–1228. Crossref, Medline, Google Scholar24. Pu J, Leader JK, Bandos A, et al. Automated quantification of COVID-19 severity and progression using chest CT images. Eur Radiol 2020. 10.1007/s00330-020-07156-2. Published online August 13, 2020. Crossref, Google Scholar25. Su Y, Han Y, Liu J, et al. Tailoring steroids in the treatment of COVID-19 pneumonia assisted by CT scans: three case reports. J XRay Sci Technol 2020;28(5):885–892. Crossref, Medline, Google Scholar26. Mushtaq J, Pennella R, Lavalle S, et al. Initial chest radiographs and artificial intelligence (AI) predict clinical outcomes in COVID-19 patients: analysis of 697 Italian patients. Eur Radiol 2020. 10.1007/s00330-020-07269-8. Published online September 18, 2020. Crossref, Google Scholar27. Hao B, Sotudian S, Wang T, et al. Early prediction of level-of-care requirements in patients with COVID-19. eLife 2020;9:e60519. Crossref, Medline, Google Scholar28. Zhang R, Tie X, Qi Z, et al. Diagnosis of COVID-19 Pneumonia Using Chest Radiography: Value of Artificial Intelligence. Radiology 2020. 10.1148/radiol.2020202944. Published online September 24, 2020. Link, Google Scholar29. Bressem KK, Adams LC, Erxleben C, Hamm B, Niehues SM, Vahldiek JL. Comparing different deep learning architectures for classification of chest radiographs. Sci Rep 2020;10(1):13590. Crossref, Medline, Google Scholar30. Wang M, Xia C, Huang L, et al. Deep learning-based triage and analysis of lesion burden for COVID-19: a retrospective study with external validation. Lancet Digit Health 2020;2(10):e506–e515. Crossref, Medline, Google Scholar31. Wang J, Bao Y, Wen Y, et al. Prior-Attention Residual Learning for More Discriminative COVID-19 Screening in CT Images. IEEE Trans Med Imaging 2020;39(8):2572–2583. Crossref, Medline, Google Scholar32. Fan DP, Zhou T, Ji GP, et al. Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images. IEEE Trans Med Imaging 2020;39(8):2626–2637. Crossref, Medline, Google Scholar33. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells W, Frangi A, eds. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Cham, Switzerland: Springer, 2015; 234–241. Crossref, Google Scholar34. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, July 21–26, 2017. Piscataway, NJ: IEEE, 2017; 3462–3471. Crossref, Google Scholar35. Yan K, Wang X, Lu L, Summers RM. DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J Med Imaging (Bellingham) 2018;5(3):036501. Medline, Google Scholar36. Mesas AE, Cavero-Redondo I, Álvarez-Bueno C, et al. Predictors of in-hospital COVID-19 mortality: A comprehensive systematic review and meta-analysis exploring differences by age, sex and health conditions. PLoS One 2020;15(11):e0241742. Crossref, Medline, Google Scholar37. Gupta A, Madhavan MV, Sehgal K, et al. Extrapulmonary manifestations of COVID-19. Nat Med 2020;26(7):1017–1032. Crossref, Medline, Google Scholar38. Bhayana R, Som A, Li MD, et al. Abdominal Imaging Findings in COVID-19: Preliminary Observations. Radiology 2020;297(1):E207–E215. Link, Google ScholarArticle HistoryReceived: Nov 6 2020Revision requested: Nov 10 2020Revision received: Nov 10 2020Accepted: Nov 13 2020Published online: Dec 22 2020Published in print: Mar 2021 FiguresReferencesRelatedDetailsCited ByARTIFICIAL INTELLIGENCE IN TACKLING CORONAVIRUS AND FUTURE PANDEMICSShaguftaQuazi, Sampa KarmakarSingh, Rudra PrasadSaha, ArpitaDas, Manoj KumarSingh15 March 2024 | Journal of Experimental Biology and Agricultural Sciences, Vol. 12, No. 1AI and machine learning ethics, law, diversity, and global impactKatherineDrabiak, SkylarKyzer, ValerieNemov, IssamEl Naqa2023Oct1 | The British Journal of Radiology, Vol. 96, No. 1150Impact of Artificial Intelligence on Human Capital in Healthcare Sector Post-COVID-19AmrinderSingh, GeetikaMadaan, H RSwapna, AnujKumar2023Feb10Integrated Bayesian and association-rules methods for autonomously orienting COVID-19 patientsAdelThaljaoui, Salim ElKhediri, EmnaBenmohamed, AbdulatifAlabdulatif, AbdullahAlourani7 October 2022 | Medical & Biological Engineering & Computing, Vol. 60, No. 12Assessing Trustworthy AI in Times of COVID-19: Deep Learning for Predicting a Multiregional Score Conveying the Degree of Lung Compromise in COVID-19 PatientsHimanshiAllahabadi, JuliaAmann, IsabelleBalot, AndreaBeretta, CharlesBinkley, JonasBozenhard, FrederickBruneault, JamesBrusseau, SemaCandemir, Luca AlessandroCappellini, SubrataChakraborty, NicoletaCherciu, ChristinaCociancig, MeganCoffee, IreneEk, LeonardoEspinosa-Leal, DavideFarina, GenevieveFieux-Castagnet, ThomasFrauenfelder, AlessioGallucci, GuyaGiuliani, AdamGolda, Irmhildvan Halem, ElisabethHildt, SuneHolm, GeorgiosKararigas, Sebastien A.Krier, UlrichKuhne, FrancescaLizzi, Vince I.Madai, Aniek F.Markus, SergMasis, Emilie WiinbladMathez, FrancescoMureddu, EmanueleNeri, WalterOsika, MatissOzols, CeciliaPanigutti, BrendanParent, FrancescaPratesi, Pedro A.Moreno-Sanchez, GiovanniSartor, MattiaSavardi, AlbertoSignoroni, Hanna-MariaSormunen, AndySpezzatti, AdarshSrivastava, Annette F.Stephansen, Lau BeeTheng, Jesmin JahanTithi, JarnoTuominen, StevenUmbrello, FilippoVaccher, DennisVetter, MagnusWesterlund, ReneeWurth, Roberto V.Zicari2022Dec1 | IEEE Transactions on Technology and Society, Vol. 3, No. 4A comparison of Covid-19 early detection between convolutional neural networks and radiologistsAlbertoAlbiol, FranciscoAlbiol, RobertoParedes, Juana MaríaPlasencia-Martínez, AnaBlanco Barrio, José M. GarcíaSantos, SalvadorTortajada, Victoria M.González Montaño, Clara E.Rodríguez Godoy, SarayFernández Gómez, ElenaOliver-Garcia, Maríade la Iglesia Vayá, Francisca L.Márquez Pérez, Juan I.Rayo Madrid28 July 2022 | Insights into Imaging, Vol. 13, No. 1A novel intelligent radiomic analysis of perfusion SPECT/CT images to optimize pulmonary embolism diagnosis in COVID-19 patientsSoniaBaeza, DeboraGil, IgnasiGarcia-Olivé, MaiteSalcedo-Pujantell, JordiDeportós, CarlesSanchez, GuillermoTorres, GloriaMoragas, AntoniRosell5 December 2022 | EJNMMI Physics, Vol. 9, No. 1Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping ReviewBardiaKhosravi, PouriaRouzrokh, ShahriarFaghani, ManaMoassefi, SanazVahdati, ElhamMahmoudi, HamidChalian, Bradley J.Erickson17 October 2022 | Diagnostics, Vol. 12, No. 10Artificial intelligence to evaluate diagnosed COVID-19 chest radiographsBrunoTakara, FelipeFreitas, AlexandreBacelar, MirkoSalomon Alva Sanchez, RochelleLykawka18 September 2022 | Brazilian Journal of Radiation Sciences, Vol. 10, No. 3Mortality Prediction Analysis among COVID-19 Inpatients Using Clinical Variables and Deep Learning Chest Radiography Imaging FeaturesXuan V.Nguyen, EnginDikici, SemaCandemir, Robyn L.Ball, Luciano M.Prevedello13 July 2022 | Tomography, Vol. 8, No. 4COVID-19 prognostic modeling using CT radiomic features and machine learning algorithms: Analysis of a multi-institutional dataset of 14,339 patientsIsaacShiri, YazdanSalimi, MasoumehPakbin, GhasemHajianfar, Atlas HaddadiAvval, AmirhosseinSanaat, ShayanMostafaei, AzadehAkhavanallaf, AbdollahSaberi, ZahraMansouri, DariushAskari, MohammadrezaGhasemian, EhsanSharifipour, SalehSandoughdaran, AhmadSohrabi, ElhamSadati, SomayehLivani, PooyaIranpour, ShahriarKolahi, MaziarKhateri, SalarBijari, Mohammad RezaAtashzar, Sajad P.Shayesteh, BardiaKhosravi, Mohammad RezaBabaei, ElnazJenabi, MohammadHasanian, AlirezaShahhamzeh, Seyaed YaserForoghi Ghomi, AbolfazlMozafari, ArashTeimouri, FatemehMovaseghi, AzinAhmari, NedaGoharpey, RamaBozorgmehr, HesamaddinShirzad-Aski, RoozbehMortazavi, JalalKarimi, NazaninMortazavi, SimaBesharat, MandanaAfsharpad, HamidAbdollahi, ParhamGeramifar, Amir RezaRadmard, HosseinArabi, KiaraRezaei-Kalantari, MehrdadOveisi, ArmanRahmim, HabibZaidi2022Jun1 | Computers in Biology and Medicine, Vol. 145Virtual versus reality: external validation of COVID-19 classifiers using XCAT phantoms for chest computed tomographyFakrul IslamTushar, EhsanAbadi, SamanSotoudeh-Paima, Rafael B.Fricks, Maciej A.Mazurowski, William P.Segars, EhsanSamei, Joseph Y.Lo, Khan M.Iftekharuddin, KarenDrukker, Maciej A.Mazurowski, HongbingLu, ChisakoMuramatsu, Ravi K.Samala2022Apr4Let AI Perform Better Next Time—A Systematic Review of Medical Imaging-Based Automated Diagnosis of COVID-19: 2020–2022FanLiu, DelongChen, XiaocongZhou, WenwenDai, FengXu12 April 2022 | Applied Sciences, Vol. 12, No. 8Long-COVID diagnosis: From diagnostic to advanced AI-driven modelsRiccardoCau, GavinoFaa, ValentinaNardi, AntonellaBalestrieri, JosepPuig, Jasjit SSuri, RobertoSanFilippo, LucaSaba2022Mar1 | European Journal of Radiology, Vol. 148The Role of Artificial Intelligence and Data Science Against COVID-19SaumyaTripathi, Dilip KumarSharma20 April 2022The role of artificial intelligence in plain chest radiographs interpretation during the Covid-19 pandemicDanaAlNuaimi, ReemAlKetbi2022Jan1 | BJR|Open, Vol. 4, No. 1Current limitations to identify covid-19 using artificial intelligence with chest x-ray imaging (part ii). The shortcut learning problemJosé DanielLópez-Cabrera, RubénOrozco-Morales, Jorge ArmandoPortal-Díaz, OrlandoLovelle-Enríquez, MarlénPérez-Díaz10 October 2021 | Health and Technology, Vol. 11, No. 6Clinical Artificial Intelligence Applications in RadiologySungwonLee, Ronald M.Summers2021Nov1 | Radiologic Clinics of North America, Vol. 59, No. 6Artificial intelligence-driven assessment of radiological images for COVID-19YassineBouchareb, PegahMoradi Khaniabadi, FaizaAl Kindi, HumoudAl Dhuhli, IsaacShiri, HabibZaidi, ArmanRahmim2021Sep1 | Computers in Biology and Medicine, Vol. 136Machine Learning for COVID-19 Diagnosis and Prognostication: Lessons for Amplifying the Signal While Reducing the NoiseDerek Driggs, Ian Selby, Michael Roberts, Effrossyni Gkrania-Klotsas, James H. F. Rudd, Guang Yang, Judith Babar, Evis Sala, Carola-Bibiane Schönlieb, 24 March 2021 | Radiology: Artificial Intelligence, Vol. 3, No. 4Imaging spectrum of abdominal manifestations of COVID-19TanviVaidya, AvinashNanivadekar, RajanPatel2021Jun28 | World Journal of Radiology, Vol. 13, No. 6Lessons from the COVID-19 Pandemic on the Use of Artificial Intelligence in Digital Radiology: The Submission of a Survey to Investigate the Opinion of InsidersDanieleGiansanti, IvanoRossi, LisaMonoscalco15 March 2021 | Healthcare, Vol. 9, No. 3Recommended Articles How Might AI and Chest Imaging Help Unravel COVID-19's Mysteries?Radiology: Artificial Intelligence2020Volume: 2Issue: 3Atypical Pneumonia: Definition, Causes, and Imaging FeaturesRadioGraphics2021Volume: 41Issue: 3pp. 720-741CT Radiomics, Radiologists, and Clinical Information in Predicting Outcome of Patients with COVID-19 PneumoniaRadiology: Cardiothoracic Imaging2020Volume: 2Issue: 4Natural Language Processing and Machine Learning for Detection of Respiratory Illness by Chest CT Imaging and Tracking of COVID-19 Pandemic in the United StatesRadiology: Cardiothoracic Imaging2021Volume: 3Issue: 1Seeing Is Believing: COVID-19 Vaccination Leads to Less Pneumonia at Chest CTRadiology2022Volume: 303Issue: 3pp. 693-695See More RSNA Education Exhibits 360-Degree Review Of Lessons Learned From The COVID-19 Pandemic: The Spectrum Of Imaging Findings, Impact On Radiology Departments And The Role Of Artificial IntelligenceDigital Posters2020A Comprehensive Imaging Review of COVID-19 Pneumonia with Focus on RSNA Expert Consensus, Fleischner Society Statement and ACR Recommendations: Challenges, Opportunities and Future Direction Digital Posters2020The Role of Radiology in Disease Outbreaks Throughout HistoryDigital Posters2020 RSNA Case Collection Coronavirus (COVID-19) pneumoniaRSNA Case Collection2020COVID 19 pneumonia RSNA Case Collection2020COVID-19 pneumonia and pulmonary embolismRSNA Case Collection2020 Vol. 298, No. 3 Metrics Altmetric Score Open AccessThis article is made available via the PMC Open Access Subset for unrestricted re-use and analyses in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the COVID-19 pandemic or until permissions are revoked in writing. Upon expiration of these permissions, PMC is granted a perpetual license to make this article available via PMC and Europe PMC, consistent with existing copyright protections.PDF download