Deep Reinforcement Learning Method for 3D-CT Nasopharyngeal Cancer Localization with Prior Knowledge
Source abstract
Abstract
Fast and accurate lesion localization is an important step in medical image analysis. The current supervised deep learning methods have obvious limitations in the application of radiology, as they require a large number of manually annotated images. In response to the above issues, we introduced a deep reinforcement learning (DRL)-based method to locate nasopharyngeal carcinoma lesions in 3D-CT scans. The proposed method uses prior knowledge to guide the agent to reasonably reduce the search space and promote the convergence rate of the model. Furthermore, the multi-scale processing technique is also used to promote the localization of small objects. We trained the proposed model with 3D-CT scans of 50 patients and evaluated it with 3D-CT scans of 30 patients. The experimental results showed that the proposed model has strong robustness, and its accuracy was improved by more than 1 mm on average under the premise of using a smaller dataset compared with the DQN models in recent studies. The proposed model could effectively locate the lesion area of nasopharyngeal carcinoma in 3D-CT scans.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Brain Tumor Detection and Classification · Medical Image Segmentation Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Engineering, North China University of Water Resources and Electric Power, Zhengzhou 450046, China
School of Biomedical Engineering, Sun Yat-sen University, Shenzhen 518107, China
State Key Laboratory of Oncology in South China, Sun Yat-sen University Cancer Center, Guangzhou 510060, China
Funding
National Natural Science Foundation of China · Shenzhen Fundamental Research Program of China · High-level Talents Research Project of NCWU
References (34)
Show source-provided references
1. Goodfellow, I.J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv.
2. Analysis of explainers of black box deep neural networks for computer vision: A survey10.3390/make3040048 ↗
3. Stember, J., and Shalu, H. (2020). Deep reinforcement learning to detect brain lesions on MRI: A proof-of-concept application of reinforcement learning to medical images. arXiv.
4. Cancer survival in China, 2003–2005: A population-based study10.1002/ijc.29227 ↗
5. Synergistic Association of Hepatitis B Surface Antigen and Plasma Epstein-Barr Virus DNA Load on Distant Metastasis in Patients with Nasopharyngeal Carcinoma10.1001/jamanetworkopen.2022.53832 ↗
6. Criminisi, A., Shotton, J., Robertson, D., and Konukoglu, E. (2010, January 20). Regression forests for efficient anatomy detection and localization in ct studies. Proceedings of the International MICCAI Workshop on Medical Computer Vision, Beijing, China.10.1007/978-3-642-18421-5_11 ↗
7. Gauriau, R., Cuingnet, R., Lesage, D., and Bloch, I. (2014, January 14–18). Multi-organ localization combining global-to-local regression and confidence maps. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Boston, MA, USA.10.1007/978-3-319-10443-0_43 ↗
8. Ma, K., Wang, J., Singh, V., Tamersoy, B., Chang, Y.-J., Wimmer, A., and Chen, T. (2017, January 11–13). Multimodal image registration with deep context reinforcement learning. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Quebec City, QC, Canada.10.1007/978-3-319-66182-7_28 ↗
9. Jie, Z., Liang, X., Feng, J., Jin, X., Lu, W., and Yan, S. (2016). Tree-Structured Reinforcement Learning for Sequential Object Localization. arXiv.
10. Maicas, G., Carneiro, G., Bradley, A.P., Nascimento, J.C., and Reid, I. (2017, January 11–13). Deep reinforcement learning for active breast lesion detection from dce-mri. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Quebec City, QC, Canada.10.1007/978-3-319-66179-7_76 ↗
11. Dou, H., Yang, X., Qian, J., Xue, W., and Ni, D. (2019, January 13–17). Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China.10.1007/978-3-030-32254-0_33 ↗
12. Ghesu, F.C., Georgescu, B., Mansi, T., Neumann, D., Hornegger, J., and Comaniciu, D. (2016, January 17–21). An artificial agent for anatomical landmark detection in medical images. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece.10.1007/978-3-319-46726-9_27 ↗
13. Zhang, M., Xu, J., Abaci Turk, E., Grant, P.E., Golland, P., and Adalsteinsson, E. (2020, January 4–8). Enhanced detection of fetal pose in 3D MRI by deep reinforcement learning with physical structure priors on anatomy. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru.10.1007/978-3-030-59725-2_38 ↗
14. Pesce, E., Ypsilantis, P.P., Withey, S., Bakewell, R., Goh, V., and Montana, G. (2017). Learning to detect chest radiographs containing lung nodules using visual attention networks. arXiv.
15. Zhang, P., Wang, F., and Zheng, Y. (2018, January 16–20). Deep Reinforcement Learning for Vessel Centerline Tracing in Multi-modality 3D Volumes. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.10.1007/978-3-030-00937-3_86 ↗
16. Stember, J.N., and Shalu, H. (2022). Reinforcement learning using Deep Q Networks and Q learning accurately localizes brain tumors on MRI with very small training sets. BMC Med. Imaging, 22.10.1186/s12880-022-00919-x ↗
17. Navarro, F., Sekuboyina, A., Waldmannstetter, D., Peeken, J.C., Combs, S.E., and Menze, B.H. (2020, January 6–9). Deep reinforcement learning for organ localization in CT. Proceedings of the Third Conference on Medical Imaging with Deep Learning, Montréal, QC, Canada.
18. Jain, A., Powers, A., and Johnson, H.J. (2020, January 3–7). Robust automatic multiple landmark detection. Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA.10.1109/ISBI45749.2020.9098329 ↗
19. Deep reinforcement learning with its application for lung cancer detection in medical Internet of Things10.1016/j.future.2019.02.068 ↗
20. Lung nodule detection via deep reinforcement learning10.3389/fonc.2018.00108 ↗
21. Alansary, A., Le Folgoc, L., Vaillant, G., Oktay, O., Li, Y., Bai, W., Passerat-Palmbach, J., Guerrero, R., Kamnitsas, K., and Hou, B. (2018, January 16–20). Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.10.1007/978-3-030-00928-1_32 ↗
22. Partial Policy-Based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images10.1109/TMI.2019.2946345 ↗
23. Ghesu, F.C., Georgescu, B., Grbic, S., Maier, A.K., Hornegger, J., and Comaniciu, D. (2017, January 11–13). Robust Multi-scale Anatomical Landmark Detection in Incomplete 3D-CT Data. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Quebec City, QC, Canada.10.1007/978-3-319-66182-7_23 ↗
24. Multi-scale deep reinforcement learning for real-time 3D-landmark detection in CT scans10.1109/TPAMI.2017.2782687 ↗
25. Leroy, G., Rueckert, D., and Alansary, A. (2020, January 4–8). Communicative Reinforcement Learning Agents for Landmark Detection in Brain Images. Proceedings of the Machine Learning in Clinical Neuroimaging and Radiogenomics in Neuro-Oncology: Third International Workshop, MLCN 2020, and Second International Workshop, RNO-AI, Lima, Peru.10.1007/978-3-030-66843-3_18 ↗
26. Bellman, R. (2013). Dynamic Programming, Courier Corporation.
27. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
28. Towards Intelligent Robust Detection of Anatomical Structures in Incomplete Volumetric Data10.1016/j.media.2018.06.007 ↗
29. Evaluating reinforcement learning agents for anatomical landmark detection10.1016/j.media.2019.02.007 ↗
30. Vlontzos, A., Alansary, A., Kamnitsas, K., Rueckert, D., and Kainz, B. (2019, January 13–17). Multiple landmark detection using multi-agent reinforcement learning. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China.10.1007/978-3-030-32251-9_29 ↗
31. Deep Q learning driven CT pancreas segmentation with geometry-aware U-Net10.1109/TMI.2019.2911588 ↗
32. Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized Experience Replay. arXiv.
33. User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability10.1016/j.neuroimage.2006.01.015 ↗
34. National Health Commission of the PRC, Ministry of Education, Ministry of Science and Technology, and Bureau of Traditional Chinese Medicine (2023, April 01). Measures for Ethical Review of Life Sciences and Medical Research Involving Human Subjects, Available online: https://www.gov.cn/zhengce/zhengceku/2023-02/28/content_5743658.htm.