journal article·2024-10-08Open access

Combination Pattern Method Using Deep Learning for Pill Classification

Svetlana Kim, Eun-Young Park, Jun-Seok Kim, Sun-Young Ihm

Applied Sciences· Volume 14· Issue 19

Abstract

The accurate identification of pills is essential for their safe administration in the medical field. Despite technological advancements, pill classification encounters hurdles such as ambiguous images, pattern similarities, mixed pills, and variations in pill shapes. A significant factor is the inability of 2D imaging to capture a pill’s 3D structure efficiently. Additionally, the scarcity of diverse datasets reflecting various pill shapes and colors hampers accurate prediction. Our experimental investigation shows that while color-based classification obtains a 95% accuracy rate, shape-based classification only reaches 66%, underscoring the inherent difficulty distinguishing between pills with similar patterns. In response to these challenges, we propose a novel system integrating Multi Combination Pattern Labeling (MCPL), a new method designed to accurately extract feature points and pill patterns. MCPL extracts feature points invariant to rotation and scale and effectively identifies unique edges, thereby emphasizing pills’ contour and structural features. This innovative approach enables the robust extraction of information regarding various shapes, sizes, and complex pill patterns, considering even the 3D structure of the pills. Experimental results show that the proposed method improves the existing recognition performance by about 1.2 times. By improving the accuracy and reliability of pill classification and recognition, MCPL can significantly enhance patient safety and medical efficiency. By overcoming the limitations inherent in existing classification methods, MCPL provides high-accuracy pill classification, even with constrained datasets. It substantially enhances the reliability of pill classification and recognition, contributing to improved patient safety and medical efficiency.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Currency Recognition and Detection · Cold Fusion and Nuclear Reactions · Dental Radiography and Imaging

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of AI Convergence & Engineering, Open Cyber University of Korea, C-9F, 353, Mangu-ro, Jungnang-gu, Seoul 02087, Republic of Korea

Department of Visual Design, Hyupsung University, 72, Choerubaek-ro, Bongdam-eup, Hwaseong-si 18330, Republic of Korea

Department of Computer Engineering, Pai Chai University, 155-40 Baejae-ro, Seo-gu, Daejeon 35345, Republic of Korea

Funding

National Research Foundation of Korea (NRF) · Korea government (MSIT) · Innovative Human Resource Development for Local Intellectualization · Institute of Information & Communications Technology Planning & Evaluation(IITP)

References (47)

Show source-provided references
1. Slawomirski, L., Auraaen, A., and Klazinga, N. (2017). The Economics of Patient Safety: Strengthening a Value-Based Approach to Reducing Patient Harm at National Level, OECD. OECD Health Working Papers, no.96.
2. Medication errors: A focus on nursing practice10.1093/jphsr/rmaa025 ↗
3. Hospital medication errors: A cross-sectional study10.1093/intqhc/mzaa136 ↗
4. Prevalence, contributing factors, and interventions to reduce medication errors in outpatient and ambulatory settings: A systematic review10.1007/s11096-023-01626-5 ↗
5. Causes of Medication Administration Errors in Hospitals: A Systematic Review of Quantitative and Qualitative Evidence10.1007/s40264-013-0090-2 ↗
6. Tefera, Y.G., Gebresillassie, B.M., Ayele, A.A., Belay, A.B., and Emiru, A.K. (2019). The characteristics of drug information inquiries in an Ethiopian university hospital: A two-year observational study. Sci. Rep., 9.10.1038/s41598-019-50204-1 ↗
7. Assessment of queries received by the drug information center at King Saud Medical City10.4103/JPBS.JPBS_166_17 ↗
8. Object tracking using adaptive template matching10.5573/IEIESPC.2015.4.1.001 ↗
9. Automated visual inspection of glass bottle bottom with saliency detection and template matching10.1109/TIM.2018.2886977 ↗
10. Ushizima, D., Carneiro, A., Souza, M., and Medeiros, F. (2015, January 14–16). Investigating pill recognition methods for a new national library of medicine image dataset. Proceedings of the ISVC 2015, Las Vegas, NV, USA.10.1007/978-3-319-27863-6_38 ↗
11. Ling, S., Pastor, A., Li, J., Che, Z., Wang, J., Kim, J., and Callet, P.L. (2020, January 13–19). Few-Shot Pill Recognition. Proceedings of the CVPR 2020, Seattle, WA, USA.10.1109/CVPR42600.2020.00981 ↗
12. Yaniv, Z., Faruque, J., Howe, S., Dunn, K., Sharlip, D., Bond, A., Perillan, P., Bodenreider, O., Ackerman, M.J., and Yoo, T.S. (2016, January 18–20). The national library of medicine pill image recognition challenge: An initial report. Proceedings of the IEEE Applied Imagery Pattern Recognition Workshop, Washington, DC, USA.10.1109/AIPR.2016.8010584 ↗
13. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
14. Deep convolutional neural networks for accurate somatic mutation detection10.1038/s41467-019-09027-x ↗
15. Dermatologist-level classification of skin cancer with deep neural networks10.1038/nature21056 ↗
16. Identifying medical diagnoses and treatable diseases by image-based deep learning10.1016/j.cell.2018.02.010 ↗
17. Deep learning: New computational modelling techniques for genomics10.1038/s41576-019-0122-6 ↗
18. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
19. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
20. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the CVPR 2015, Santiago, Chile.10.1109/ICCV.2015.169 ↗
21. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
22. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the CVPR 2016, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
23. Fast and accurate medication identification10.1038/s41746-019-0086-0 ↗
24. ImageNet classification with deep convolutional neural networks10.1145/3065386 ↗
25. Development of fine-grained pill identification algorithm using deep convolutional network10.1016/j.jbi.2017.09.005 ↗
26. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 8–10). Going deeper with convolutions. Proceedings of the CVPR 2015, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
27. CBIR using features derived by deep learning10.1145/3470568 ↗
28. He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the ICCV 2017, Venice, Italy.10.1109/ICCV.2017.322 ↗
29. Yao, J., Qi, J., Zhang, J., Shao, H., Yang, J., and Li, X. (2021). A real-time detection algorithm for kiwifruit defects based on YOLOv5. Electronics, 10.10.3390/electronics10141711 ↗
30. Zhao, J., Zhang, X., Yan, J., Qiu, X., Yao, X., Tian, Y., Zhu, Y., and Cao, W. (2021). A wheat spike detection method in UAV images based on improved YOLOv5. Remote Sens., 13.10.3390/rs13163095 ↗
31. Darma, I.W.A.S., Suciati, N., and Siahaan, D. (2021, January 24–25). A Performance Comparison of Balinese Carving Motif Detection and Recognition using YOLOv5 and Mask R-CNN. Proceedings of the ICICoS 2021, Semarang, Indonesia.10.1109/ICICoS53627.2021.9651855 ↗
32. Zhou, F., Zhao, H., and Nie, Z. (2021, January 22–24). Safety helmet detection based on YOLOv5. Proceedings of the ICPECA 2021, Shenyang, China.10.1109/ICPECA51329.2021.9362711 ↗
33. Robust image retrieval based on color histogram of local feature regions10.1007/s11042-009-0362-0 ↗
34. Pill-ID: Matching and retrieval of drug pill images10.1016/j.patrec.2011.08.022 ↗
35. Guo, P., Stanley, R.J., Cole, J.G., Hagerty, J., and Stoecker, W.V. (March, January 27). Color Feature-based Pillbox Image Color Recognition. Proceedings of the VISIGRAPP 2017, Porto, Portugal.10.5220/0006136001880194 ↗
36. Alcantarilla, P.F., Bartoli, A., and Davison, A.J. (2012, January 7–13). KAZE features. Proceedings of the ECCV 2012, Florence, Italy.10.1007/978-3-642-33783-3_16 ↗
37. Bay, H., Tuytelaars, T., and Gool, L.V. (2006, January 7–13). SURF: Speeded up Robust Features. Proceedings of the ECCV 2006, Graz, Austria.10.1007/11744023_32 ↗
38. Viswanathan, D.B. (2009, January 6–8). Features from accelerated segment test (fast). Proceedings of the WIAMIS 2009, London, UK.
39. Distinctive image features from scale-invariant key points10.1023/B:VISI.0000029664.99615.94 ↗
40. Szeliski, R. (2022). Computer Vision: Algorithms and Applications Second Edition, Springer Nature.10.1007/978-3-030-34372-9 ↗
41. Harris, C., and Stephens, M. (September, January 31). A combined corner and edge detector. Proceedings of the Alvey Vision Conference 1988, Manchester, UK.
42. Kwon, H.J., Kim, H.G., and Lee, S.H. (2022). Pill Detection Model for Medicine Inspection Based on Deep Learning. Chemosensors, 10.10.3390/chemosensors10010004 ↗
43. Holtkötter, J., Amaral, R., Almeida, R., Jácome, C., Cardoso, R., Pereira, A., Pereira, M., Chon, K.H., and Fonseca, J.A. (2022). Development and Validation of a Digital Image Processing–Based Pill Detection Tool for an Oral Medication Self-Monitoring System. Sensors, 22.10.20944/preprints202202.0204.v1 ↗
44. Nguyen, A.D., Pham, H.H., Trung, H.T., Nguyen, Q.V.H., Truong, T.N., and Nguyen, P.L. (2023). Highly Accurate and Explainable Multi-Pill Detection Framework with Graph Neural Network–Assisted Multimodal Data Fusion. arXiv.10.1371/journal.pone.0291865 ↗
45. A Deep Learning–Based Intelligent Medicine Recognition System for Chronic Patients10.1109/ACCESS.2019.2908843 ↗
46. A drug identification model developed using deep learning technologies: Experience of a medical center in Taiwan
47. Nguyen, A.D., Nguyen, T.D., Pham, H.H., Nguyen, T.H., and Nguyen, P.L. (2022, January 28–30). Image-based Contextual Pill Recognition with Medical Knowledge Graph Assistance. Proceedings of the ACIIDS 2022, Ho Chi Minh City, Vietnam.10.1007/978-981-19-8234-7_28 ↗