AI-Powered Service Robotics for Independent Shopping Experiences by Elderly and Disabled People
Source abstract
Abstract
Through human development and technological expansion, it has become apparent that the potential lies within each individual to have an essential part in the transcendence of society and the community. People less privileged than others may need more strength and determination to surpass their current resources to overcome normal and natural obstacles in order to simulate an environment where productivity and creativity exist. This paper aims to study an approach that will assist the elderly and people of determination in one of the most essential activities practiced by individuals: shopping. The study focuses on facilitating the acquirement of items from shelves and skipping the cashier line. The proposed system is a service robot supported by a robotic arm and a linear actuator as a lifting mechanism, controlled by a remote joystick to help the elderly or disabled people reach items on high shelves. The scanning system is based on barcode detection, using transfer learning. The network was designed using YOLOv2 layers connected to TinyYOLO as feature extraction layers. This network has proven to be the most practical, with 86.4% accuracy and real-time operation with 27 FPS in comparison to using the YOLOv2 layers with DarkNet or VGG19 as feature extraction layers. An anti-theft system is integrated into the robot to improve the reliability of the self-checkout feature. The system uses computer vision GMM and Kalman filter for item detection inside the cart, and the item is validated to be the one that has been scanned, using SURF for structural features, HSV for color, and load-sensors mounted to the base of the cart to measure the item’s weight.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: IoT and Edge/Fog Computing · Advanced Manufacturing and Logistics Optimization · Robotics and Automated Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Electrical, Computer and Biomedical Engineering Department, College of Engineering, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates
Mechanical Engineering Department, College of Engineering, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates
Bioengineering Department, University of Louisville, Louisville, KY 40292, USA
Funding
Office of Research and Sponsored Programs in Abu Dhabi University
References (27)
Show source-provided references
1. Al Youm, E. (2021, September 17). Obstacles Faced by People of Determination. Available online: https://www.emaratalyoum.com/business/local/2019-06-10-1.1221161.
2. Winlock, T., Christiansen, E., and Belongie, S. (2010, January 13–18). Toward real-time grocery detection for the visually impaired. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition—Workshops, San Francisco, CA, USA.10.1109/CVPRW.2010.5543576 ↗
3. Enhancing product detection with multicue optimization for tv shopping applications10.1109/TETC.2014.2386140 ↗
4. Belibov, D., and Tudose, D.S. (2019, January 10–12). Smart shopping cart. Proceedings of the 18th RoEduNet Conference: Networking in Education and Research (RoEduNet), Galați, Romania.10.1109/ROEDUNET.2019.8909680 ↗
5. Alves, R., Admas Linhares, B., and Souza, J.R. (2018, January 6–10). Autonomous Shopping Cart: A New Concept of Service Robot for Assisting Customers. Proceedings of the 2018 Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE), Paraiba, Brazil.10.1109/LARS/SBR/WRE.2018.00086 ↗
6. Arciuolo, T., and Abuzneid, A. (2019, January 5–7). Simultaneously Shop, Bag, and Checkout (2SBC-Cart): A Smart Cart for Expedited Supermarket Shopping. Proceedings of the 2019 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.10.1109/CSCI49370.2019.00219 ↗
7. Subudhi, S.R., and Ponnalagu, R.N. (2019, January 13–15). An Intelligent Shopping Cart with Automatic Product Detection and Secure Payment System. Proceedings of the 2019 IEEE 16th India Council International Conference (INDICON), Rajkot, India.10.1109/INDICON47234.2019.9030331 ↗
8. Mekruksavanich, S. (2019, January 18–20). The Smart Shopping Basket Based on IoT Applications. Proceedings of the 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China.10.1109/ICSESS47205.2019.9040750 ↗
9. Sarwar, M.A., Daraghmi, Y., Liu, K., Chi, H., İk, T.U., and Li, Y. (2020, January 25–28). Smart Shopping Carts Based on Mobile Computing and Deep Learning Cloud Services. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference (WCNC), Seoul, Korea.10.1109/WCNC45663.2020.9120574 ↗
10. Popa, M., Rothkrantz, L., Yang, Z., Wiggers, P., Popa, M., Braspenning, R., and Shan, C. (2010, January 10–13). Analysis of shopping behavior based on surveillance system. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Istanbul, Turkey.10.1109/ICSMC.2010.5641928 ↗
11. The amazon go concept: Implications, applications, and sustainability
12. Grocery product detection and recognition10.1016/j.eswa.2017.02.050 ↗
13. Zhang, H., Zhang, L., Li, P., and Gu, D. (2018, January 25–27). Yarn-dyed fabric defect detection with yolov2 based on deep convolution neural networks. Proceedings of the IEEE 7th Data Driven Control and Learning Systems Conference (DDCLS), Enshi, China.10.1109/DDCLS.2018.8516094 ↗
14. Zhao, J., and Qu, J. (2019, January 5–7). Healthy and diseased tomatoes detection based on yolov2. Proceedings of the International Conference on Human Centered Computing, Čačak, Serbia.10.1007/978-3-030-15127-0_34 ↗
15. Xu, C., and Lu, X. (2018, January 10–12). Novel Gaussian mixture model background subtraction method for detecting moving objects. Proceedings of the IEEE International Conference of Safety Produce Informatization, Chongqing, China.
16. Hardjono, B., Wibisono, A., Sina, I., Nurhadiyatna, A., Jatmiko, W., and Mursanto, P. (2013, January 13–16). Background subtraction using gaussian mixture model enhanced by hole filling algorithm (gmmhf). Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK.
17. Phung, S.L., Beghdadi, A., Bouttefroy, P.L.M., and Bouzerdoum, A. (2010, January 14–19). On the analysis of background subtraction techniques using gaussian mixture models. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, USA.
18. Feng, Z., Luo, X., Yang, T., and Kita, K. (2018, January 7–10). An object Detection System Based on YOLOv2 in Fashion Apparel. Proceedings of the 2018 IEEE 4th International Conference on Computer and Communications (ICCC), Chengdu, China.10.1109/CompComm.2018.8780944 ↗
19. Online EM Algorithm for Background Subtraction10.1016/j.proeng.2011.12.688 ↗
20. Wong, S.C., Gatt, A., Stamatescu, V., and McDonnell, M.D. (December, January 30). Understanding Data Augmentation for Classification: When to Warp?. Proceedings of the International Conference on Digital Image Computing: Techniques and Applications (DICTA), Gold Coast, Australia.
21. Mikołajczyk, A., and Grochowski, M. (2018, January 9–12). Data augmentation for improving deep learning in image classification problem. Proceedings of the International Interdisciplinary PhD Workshop (IIPhDW), Swinoujscie, Poland.10.1109/IIPHDW.2018.8388338 ↗
22. Molchanov, V.V., Vishnyakov, B.V., Vizilter, Y.V., Vishnyakova, O.V., and Knyaz, V.A. (2017). Pedestrian detection in video surveillance using fully convolutional yolo neural network. Automated Visual Inspection and Machine Vision II, International Society for Optics and Photonics.10.1117/12.2270326 ↗
23. Kingma, D.P., and Adam, J.B. (2014, January 14–16). A method for stochastic optimization. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada.
24. Bottou, L. (2010, January 22–27). Large-scale machine learning with stochastic gradient descent. Proceedings of the International Conference on Computational Statistics, Paris, France.10.1007/978-3-7908-2604-3_16 ↗
25. On the momentum term in gradient descent learning algorithms10.1016/S0893-6080(98)00116-6 ↗
26. Multiple Object Detection using GMM Technique and Tracking using Kalman Filter
27. A Review of Service Robots Coping with Uncertain Information in Natural Language Instructions10.1109/ACCESS.2018.2808369 ↗