Multiple Feature Integration for Classification of Thoracic Disease in Chest Radiography
Source abstract
Abstract
The accurate localization and classification of lung abnormalities from radiological images are important for clinical diagnosis and treatment strategies. However, multilabel classification, wherein medical images are interpreted to point out multiple existing or suspected pathologies, presents practical constraints. Building a highly precise classification model typically requires a huge number of images manually annotated with labels and finding masks that are expensive to acquire in practice. To address this intrinsically weakly supervised learning problem, we present the integration of different features extracted from shallow handcrafted techniques and a pretrained deep CNN model. The model consists of two main approaches: a localization approach that concentrates adaptively on the pathologically abnormal regions utilizing pretrained DenseNet-121 and a classification approach that integrates four types of local and deep features extracted respectively from SIFT, GIST, LBP, and HOG, and convolutional CNN features. We demonstrate that our approaches efficiently leverage interdependencies among target annotations and establish the state of the art classification results of 14 thoracic diseases in comparison with current reference baselines on the publicly available ChestX-ray14 dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: COVID-19 diagnosis using AI · Lung Cancer Diagnosis and Treatment · Radiomics and Machine Learning in Medical Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Korea
Department of Software, Korea National University of Transportation, Chungju 27469, Korea
Funding
National Research Foundation of Korea
References (62)
Show source-provided references
1. Brenner, D.R., McLaughlin, J.R., and Hung, R.J. (2011). Previous lung diseases and lung cancer risk: A systematic review and meta-analysis. PLoS ONE, 6.10.1371/journal.pone.0017479 ↗
2. Viral pneumonia10.1016/S0140-6736(10)61459-6 ↗
3. A survey on deep learning in medical image analysis10.1016/j.media.2017.07.005 ↗
4. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R.M. (2017, January 21–26). Chest X-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.369 ↗
5. Liu, C., Mao, J., Sha, F., and Yuille, A.L. (2017, January 4–9). Attention correctness in neural image captioning. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.11197 ↗
6. Ba, J., Mnih, V., and Kavukcuoglu, K. (2014). Multiple object recognition with visual attention. arXiv.
7. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
8. Simonyan, K., and Zisserman, A. (2015, January 14–16). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada.
9. Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., and Oliva, A. (2014, January 6–13). Learning deep features for scene recognition using places database. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
10. Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7–12). Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298682 ↗
11. Region-based convolutional networks for accurate object detection and segmentation10.1109/TPAMI.2015.2437384 ↗
12. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
13. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollr, P., and Zitnick, L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
14. Johnson, J., Karpathy, A., and Fei-Fei, L. (July, January 26). DenseCap: Fully convolutional localization networks for dense captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
15. Visual Genome: Connecting language and vision using crowdsourced dense image annotations10.1007/s11263-016-0981-7 ↗
16. Roth, H.R., Lu, L., Seff, A., Cherry, K.M., Hoffman, J., Wang, S., Liu, J., Turkbey, E., and Summers, R.M. (2014, January 14–18). A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention, Boston, MA, USA.10.1007/978-3-319-10404-1_65 ↗
17. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learnings10.1109/TMI.2016.2528162 ↗
18. Pulmonary nodule detection in ct images: False positive reduction using multi-view convolutional networks10.1109/TMI.2016.2536809 ↗
19. Automatic detection of cerebral microbleeds from mr images via 3d convolutional neural networks10.1109/TMI.2016.2528129 ↗
20. Jamaludin, A., Kadir, T., and Zisserman, A. (2016, January 17–21). SpineNet: Automatically pinpointing classification evidence in spinal MRIs. Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention, Athens, Greece.10.1007/978-3-319-46723-8_20 ↗
21. Dermatologist-level classification of skin cancer with deep neural networks10.1038/nature21056 ↗
22. Roth, H., Lu, L., Farag, A., Shin, H.-C., Liu, J., Turkbey, E.B., and Summers, R.M. (2015, January 5–9). DeepOrgan: Multi-level deep convolutional networks for automated pancreas segmentation. Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention, Munich, Germany.10.1007/978-3-319-24553-9_68 ↗
23. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
24. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns10.1109/TPAMI.2002.1017623 ↗
25. Dalal, N., and Triggs, B. (2005, January 20–26). Histograms of oriented gradients for human detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA.
26. Negrel, R., Picard, D., and Gosselin, P.H. (2014, January 8–20). Evaluation of second-order visual features for land-use classification. Proceedings of the 12th IEEE International Workshop on Content-Based Multimedia Indexing, Klagenfurt, Austria.10.1109/CBMI.2014.6849835 ↗
27. Scene classification based on the multifeature fusion probabilistic topic model for high spatial resolution remote sensing imagery10.1109/TGRS.2015.2435801 ↗
28. Penatti, O.A., Nogueira, K., and Santos, J.A.D. (2015, January 8–10). Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA.10.1109/CVPRW.2015.7301382 ↗
29. Histopathological image classification using discriminative feature-oriented dictionary learning10.1109/TMI.2015.2493530 ↗
30. Otálora, S., Cruz-Roa, A., Arevalo, J., Atzori, M., Madabhushi, A., Judkins, A.R., and Depeursinge, A. (2015, January 5–9). Combining unsupervised feature learning and riesz wavelets for histopathology image representation: Application to identifying anaplastic medulloblastoma. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24553-9_71 ↗
31. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery10.3390/rs71114680 ↗
32. Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014, January 8–13). How transferable are features in deep neural networks?. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
33. Van Ginneken, B., Setio, A.A., Jacobs, C., and Ciompi, F. (2015, January 16–19). Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans. Proceedings of the 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), New York, NY, USA.10.1109/ISBI.2015.7163869 ↗
34. Nanni, L., Ghidoni, S., and Brahnam, S. (2018). Ensemble of convolutional neural networks for bioimage classification. Appl. Comput. Inf.
35. Mahmood, A., Bennamoun, M., An, S., Sohel, F., Boussaid, F., Hovey, R., and Fisher, R.B. (2016, January 25–28). Coral classification with hybrid feature representations. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7532411 ↗
36. Tang, Y., Tang, Y., Sandfort, V., Xiao, J., and Summers, R.M. (2019). TUNA-Net: Task-oriented UNsupervised Adversarial Network for Disease Recognition in Cross-Domain Chest X-rays. arXiv.10.1007/978-3-030-32226-7_48 ↗
37. Li, Z., Wang, C., Han, M., Xue, Y., Wei, W., Li, L.J., and Fei-Fei, L. (2018, January 18–22). Thoracic disease identification and localization with limited supervision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00865 ↗
38. Salehinejad, H., Valaee, S., Dowdell, T., Colak, E., and Barfett, J. (2018, January 15–20). Generalization of deep neural networks for chest pathology classification in x-rays using generative adversarial networks. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, AB, Canada.10.1109/ICASSP.2018.8461430 ↗
39. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
40. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep residual learning for image recognition. arXiv.10.1109/CVPR.2016.90 ↗
41. Yao, L., Poblenz, E., Dagunts, D., Covington, B., Bernard, D., and Lyman, K. (2017). Learning to diagnose from scratch by exploiting dependencies among labels. arXiv.
42. Kumar, P., Grewal, M., and Srivastava, M.M. (2018, January 27–29). Boosted cascaded convnets for multilabel classification of thoracic diseases in chest radiographs. Proceedings of the International Conference Image Analysis and Recognition, Póvoa de Varzim, Portugal.10.1007/978-3-319-93000-8_62 ↗
43. Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., and Lungren, M.P. (2017). Radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv.
44. Wang, H., and Xia, Y. (2018). Chestnet: A deep neural network for classification of thoracic diseases on chest radiography. arXiv.
45. Wang, X., Peng, Y., Lu, L., Lu, Z., and Summers, R.M. (2018, January 18–22). Tienet: Text-image embedding network for common thorax disease classification and reporting in chest X-rays. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00943 ↗
46. Baltruschat, I.M., Nickisch, H., Grass, M., Knopp, T., and Saalbach, A. (2018). Comparison of deep learning approaches for multi-label chest X-ray classification. arXiv.10.1038/s41598-019-42294-8 ↗
47. Indexing of remote sensing images with different resolutions by multiple features10.1109/JSTARS.2012.2228254 ↗
48. Integrating multilayer features of convolutional neural networks for remote sensing scene classification10.1109/TGRS.2017.2711275 ↗
49. Modeling the shape of the scene: A holistic representation of the spatial envelope10.1023/A:1011139631724 ↗
50. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (July, January 26). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
51. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
52. Gaussian mixture discriminant analysis and sub-pixel land cover characterization in remote sensing10.1016/S0034-4257(02)00172-4 ↗
53. Weinberger, K.Q., and Saul, L.K. (2006, January 4–7). Distance metric learning for large margin nearest neighbor classification. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
54. Murphy, K.P. (2006). Naive Bayes Classifiers, University of British Columbia.
55. Multicategory proximal support vector machine classifiers10.1007/s10994-005-0463-6 ↗
56. Adaptive boosting for SAR automatic target recognition10.1109/TAES.2007.357120 ↗
57. Random forest classifier for remote sensing classification10.1080/01431160412331269698 ↗
58. Extreme learning machine for regression and multiclass classification10.1109/TSMCB.2011.2168604 ↗
59. Data-driven identification of prognostic tumor subpopulations using spatially mapped t-SNE of mass spectrometry imaging data10.1073/pnas.1510227113 ↗
60. Principal component analysis10.1002/wics.101 ↗
61. Guendel, S., Grbic, S., Georgescu, B., Liu, S., Maier, A., and Comaniciu, D. (2018, January 19–22). Learning to recognize abnormalities in chest X-rays with location-aware dense networks. Proceedings of the Iberoamerican Congress on Pattern Recognition, Madrid, Spain.10.1007/978-3-030-13469-3_88 ↗
62. The Prostate, Lung, Colorectal and Ovarian (PLCO) cancer screening trial of the National Cancer Institute: History, organization, and status10.1016/S0197-2456(00)00097-0 ↗