X-ray Imaging Meets Deep Learning
-
摘要:
作为人工智能(artificial intelligence,AI)的主流,深度学习在计算机视觉、图像多尺度特征提取领域已有所进展。2016年以来,深度学习方法在计算机断层成像(从积分特性,如线积分,实现内部结构的图像重建)方面也取得了进步。总体而言,在人工智能领域,尤其是基于人工智能的成像领域,令人兴奋的前景和挑战并存,包括准确性、鲁棒性、泛化性、可解释性等一系列问题。基于2021年8月2日SPIE Optics+Photonics上的大会邀请报告,本文介绍X射线成像和深度学习的背景,低剂量CT、稀疏数据CT、深度影像组学的代表性成果,讨论对于X射线CT、其他成像模式以及多模态成像而言,数据驱动和模型驱动方法融合带来的机会,以期显著促进精准医疗的进步。
-
图 4 模块化的深度去噪网络是在有放射科医生参与的闭环模式下训练的[9]。该网络生成不同程度的去噪图像,供放射科医生根据具体诊断任务决定最佳程度
图 5 我们最近设计的SUGAR网络针对相当稀疏的数据重建出很有潜力的初步结果[10]
-
[1] KERMELIOTIS T. X-ray voted top modern discovery[EB/OL]. Cable News Network, (2009-01-01). https://www.cnn.com/2009/WORLD/europe/11/04/xray.machine.science.museum/index.html. [2] WANG G, YE J C, DE MAN B. Deep learning for tomographic image reconstruction[J]. Nature Machine Intelligence, 2020, 2(12): 737-748. [3] LELL M M, KACHELEß M. Recent and upcoming technological developments in computed tomography: High speed, low dose, deep learning, multienergy[J]. Investigative Radiology, 2020, 55(1): 8−19. doi: 10.1097/RLI.0000000000000601 [4] MAIER A, SYBEN C, LASSER T, et al. A gentle introduction to deep learning in medical image processing[J]. Journal of Medical Physics, 2018, 29(2): 86−101. [5] SAHINER B, PEZESHK A, HADJIISKI L M, et al. Deep learning in medical imaging and radiation therapy[J]. Medical Physics, 2019, 46(1): e1−e36. DOI: 10.1002/mp.13264. [6] WANG G. A perspective on deep imaging[J]. IEEE Access 4, 2016: 8914−8924. [7] WANG G, CONG W X, YANG Q S. Tomographic image reconstruction via machine learning: US 10, 970, 887 B2[P]. (2016-06-24)[2021-04-06]. https://patents.google.com/patent/US10970887B2/en?oq=10970887. [8] WANG G, ZHANG Y, YE X J, et al. Machine learning for tomographic imaging[J]. IOP Publishing Ltd, 2019. [9] SHAN H, PADOLE A, HOMAYOUNIEH F, et al. Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction[J]. Nature Machine Intelligence, 2019, 1(6): 269−276. doi: 10.1038/s42256-019-0057-9 [10] WU W W, NIU C, EBRAHIMIAN S, et al. AI-enabled ultra-low-dose CT reconstruction[J/OL].(2021-01-01). https://arxiv.org/abs/2106.09834. [11] CHAO H, SHAN H, HOMAYOUNIEH F, et al. Deep learning predicts cardiovascular disease risks from lung cancer screening low dose computed tomography[J]. Nature Communications, 2021, 12(1): 2963. doi: 10.1038/s41467-021-23235-4 [12] RAINES K S, SALHA S, SANDBERG R L, et al. Three-dimensional structure determination from a single view[J]. Nature, 2010, 463(7278): 214-217. [13] WEI H. Fundamental limits of ‘ankylography’ due to dimensional deficiency[J]. Nature, 2011, 480(7375): E1. [14] WANG G, YU H, CONG W X, et al. Non-uniqueness and instability of ‘ankylography’[J]. Nature, 2011, 480(7375): E2-3. [15] SENIOR A W, EVANS R, JUMPER J, et al. Improved protein structure prediction using potentials from deep learning[J]. Nature, 2020, 577(7792): 706-710. [16] AKHTAR N, MIAN A. Threat of adversarial attacks on deep learning in computer vision: A survey[J]. IEEE Access 6, 2018, 14410-14430. [17] ANTUN V, RENNA F, POON C, et al. On instabilities of deep learning in image reconstruction and the potential costs of AI[J]. Proceedings of the National Academy of Sciences, 2020, 117(48): 201907377. [18] WU W W, HU D L, CONG W X, et al. Stabilizing deep tomographic reconstruction networks[J/OL]. https://arxiv.org/abs/2008.01846 (v1, v2, and v3, 2020, v4, 2021). [19] WU E, WU K, DANESHJOU R, et al. How medical AI devices are evaluated: Limitations and recommendations from an analysis of FDA approvals[J]. Nature Medicine, 2021, 27(4): 582-584. [20] CHAN K H R, YU Y D, YOU C, et al. ReduNet: A white-box deep network from the principle of maximizing rate reduction[J]. 2021. [21] FAN F L, XIONG J J, LI M Z, et al. ReduNet: A white-box deep network from the principle of maximizing rate reduction[J]. IEEE Trans Radiat Plasma Med Sci, 10.1109/TRPMS.2021.3066428 (2021). [22] SHAMSHIRBAND S, FATHI M, DEHZANGI A, et al. A review on deep learning approaches in healthcare systems: Taxonomies, challenges, and open issues[J]. Journal of Biomedical Informatics, 2021, 113: 103627. [23] MONTAVON G, SAMEK W, MÜLLER K R. Methods for interpreting and understanding deep neural networks[J]. Digital Signal Processing, 2018, 73: 1-15. [24] FAN F L, WANG G. Fuzzy logic interpretation of quadratic networks[J]. Neurocomputing, 2020, 374: 10-21. [25] FAN F F, XIONG J J, WANG G. Universal approximation with quadratic deep networks[J]. Neural Networks, 2020, 124, 383-392. [26] SABOUR S, FROSST N, HINTON G E. Dynamic routing between capsules[J]. Adv Neural Inf Process Syst, 2017-Decem, 3857-3867. [27] FAN F L, LAI R J, WANG G. Quasi-equivalence of width and depth of neural networks[EB/OL]. (2020-01-01). https://arxiv.org/abs/2002.02515. [28] NICKEL M, MURPHY K, TRESP V, et al. A review of relational machine learning for knowledge graphs[J]. Proceeding of the IEEE, 2015, 104(1): 11-33. [29] WANG Q, MAO Z, WANG B, et al. Knowledge graph embedding: A survey of approaches and applications[J]. IEEE Trans Knowl Data Eng, 2017, 29(12): 2724-2743. [30] LIU J, ZHU X X, LIU F, et al. OPT: Omni-perception pre-trainer for cross-modal understanding and generation[EB/OL]. (2021-01-01). https://arxiv.org/abs/2107.00249. [31] WANG G, ZHANG J, GAO H, et al. Towards omni-tomography: Grand fusion of multiple modalities for simultaneous interior tomography[J]. PLoS One, 2012, 7(6). [32] WANG G, KALRA M, MURUGAN V, et al. Vision 20/20: Simultaneous CT-MRI-Next chapter of multimodality imaging[J]. Medical Physics, 2015, 42(10): 5879-5889. [33] DINELEY J. Tackling the silent crisis in cancer care[EB/OL]. (2018-01-01). https://www.lindau-nobel.org/blog-tackling-the-silent-crisis-in-cancer-care-with-innovation/. -