Robotic-Based Touch Panel Test System Using Pattern Recognition Methods
Source abstract
Abstract
In this study, pattern recognition methods are applied to a five-degrees-of-freedom robot arm that can key in words on a touch screen for an automatic smartphone test. The proposed system can recognize Chinese characters and Mandarin phonetic symbols. The mechanical arm is able to perform corresponding movements and edit words on the screen. Pattern matching is based on the Red-Green-Blue (RGB) color space and is transformed to binary images for higher correct rate and geometric matching. A web camera is utilized to capture patterns on the tested smartphone screen. The proposed control scheme uses a support vector machine with a histogram of oriented gradient classifier to recognize Chinese Mandarin phonetic symbols and provide correct coordinates during the control process. The control scheme also calculates joint angles of the robot arm during the movement using the Denavit–Hartenberg parameters (D-H) model and fuzzy logic system. Fuzzy theory is applied to use the position error between the robot arm and target location then resend the command to adjust the arm’s position. From the experiments, the proposed control scheme can control the robot to press desired buttons on the tested smartphone. For Chinese Mandarin phonetic symbols, recognition accuracy of the test system can reach 90 percent.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Gaze Tracking and Assistive Technology · Hand Gesture Recognition Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Wistron Corporation, 158 Xingshan Road, Taipei 11469, Taiwan
Department of Communications, Navigation and Control Engineering, National Taiwan Ocean University, 2 Pei-Ning Road, Keelung 20224, Taiwan
References (38)
Show source-provided references
1. Application of Character Recognition to Robot Control on Smartphone Test System10.1177/1687814017693181 ↗
2. Real-Time Image Recognition and Path Tracking to Wheeled Mobile Robot for Taking an Elevator
3. Visual Recognition and Its Application to Robot Arm Control10.3390/app5040851 ↗
4. Chen, L., Guo, H., Wang, H., Chen, Y.-L., and Wu, X. (2015, January 24–26). The Visual Location of Workpiece based on Hermite Interpolation and Mapping for Robot Arms. Proceedings of the 2015 5th International Conference on Information Science and Technology (ICIST), Changsha, China.10.1109/ICIST.2015.7288962 ↗
5. Intelligent Manufacturing in the Context of Industry 4.0: A Review10.1016/J.ENG.2017.05.015 ↗
6. Chi, Z., and Xu, Q. (2015, January 7–11). Precision Control of Piezoelectric Actuator using Fuzzy Feedback Control with Inverse Hysteresis Compensation. Proceedings of the 10th IEEE International Conference on Nano/Micro Engineered and Molecular Systems, Xi’an, China.10.1109/NEMS.2015.7147414 ↗
7. Gang-Feng, L., Ying-Yong, Z., Chang-Le, L., and Ji-Hong, Y. (2015, January 6–9). Design and Optimization of Spraying Robot Arm for Hull Blocks. Proceedings of the 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China.10.1109/ROBIO.2015.7419734 ↗
8. Rivai, M., and Purwanto, D. (2015, January 20–21). Implementation of Fuzzy Logic Control in Robot Arm for Searching Location of Gas Leak. Proceedings of the 2015 International Seminar on Intelligent Technology and Its Applications (ISITIA), Surabaya, Indonesia.10.1109/ISITIA.2015.7219955 ↗
9. Pamungkas, D.S., and Ward, K. (2015, January 17–19). Immersive Teleoperation of a Robot Arm using Electro-tactile Feedback. Proceedings of the 2015 6th International Conference on Automation, Robotics and Applications (ICARA), Queenstown, New Zealand.10.1109/ICARA.2015.7081164 ↗
10. Suligoj, F., Jerbic, B., Svaco, M., Sekoranja, B., Mihalinec, D., and Vidakovic, J. (October, January 28). Medical Applicability of a Low-cost Industrial Robot Arm Guided with an Optical Tracking System. Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.10.1109/IROS.2015.7353908 ↗
11. Shih, C.-H., and Juang, J.-G. (2016). Moving Object Tracking and Its Application to an Indoor Dual-Robot Patrol. Appl. Sci., 6.10.3390/app6110349 ↗
12. Fuzzy Control Simultaneous Localization and Mapping Strategy Based on Iterative Closest Point and k-Dimensional Tree Algorithms
13. Huang, G.S., Tung, C.K., Lin, H.C., and Hsiao, S.H. (2011, January 15–18). Inverse Kinematics Analysis Trajectory Planning for a Robot Arm. Proceedings of the 2011 IEEE International Conference on Robotics & Automation, Kaohsiung, Taiwan.
14. Ardianto, S., Chen, C.-J., and Hang, H.-M. (2017, January 22–24). Real-time Traffic Sign Recognition using Color Segmentation and SVM. Proceedings of the 2017 International Conference on Systems, Signals and Image Processing (IWSSIP), Poznan, Poland.10.1109/IWSSIP.2017.7965570 ↗
15. Kumar, P., Happy, S.L., and Routray, A. (2016, January 19–21). A Real-time Robust Facial Expression Recognition System using HOG Features. Proceedings of the 2016 International Conference on Computing, Analytics and Security Trends (CAST), Pune, India.10.1109/CAST.2016.7914982 ↗
16. Document Delivery Robot Based On Image Processing and Fuzzy Control10.1139/tcsme-2016-0055 ↗
17. Boser, B.E., Guyon, I.M., and Vapnik, V.N. (1992). A Training Algorithm for Optimal Margin Classifiers, Association for Computing Machinery (ACM).10.1145/130385.130401 ↗
18. An Analytic Approach to Converting POE Parameters into D–H Parameters for Serial-Link Robots10.1109/LRA.2017.2723470 ↗
19. Multi-Lingual Optical Character Recognition System Using the Reinforcement Learning of Character Segmenter10.1109/ACCESS.2020.3025769 ↗
20. A Survey of Mono- and Multi-lingual Character Recognition using Deep and Shallow Architectures: Indic and non-indic scripts10.1007/s10462-019-09720-9 ↗
21. Deep Learning-Aided OCR Techniques for Chinese Uppercase Characters in the Application of Internet of Things10.1109/ACCESS.2019.2909401 ↗
22. Wang, X., and Lai, W. (2010, January 3–5). Edge Detection for Chinese Text Image based on Novel Differential Operator. Proceedings of the 2010 International Conference on Computer and Information Application, Tianjin, China.10.1109/ICCIA.2010.6141533 ↗
23. Liu, M., Suo, Y., and Ding, Y. (2014, January 18–20). Research on Optimization Segmentation Algorithm for Chinese/English Mixed Character Image in OCR. Proceedings of the 2014 Fourth International Conference on Instrumentation and Measurement, Computer, Communication and Control, Harbin, China.
24. Guzmán, S., Gómez, A., Diez, G., and Fernández, D. (2015, January 23–25). Car Detection Methodology in Outdoor Environment Based on Histogram of Oriented Gradient (HOG) and Support Vector Machine (SVM). Proceedings of the 6th Latin-American Conference on Networked and Electronic Media (LACNEM 2015), Medellin, Colombia.10.1049/ic.2015.0310 ↗
25. Sugiharto, A., and Harjoko, A. (2016, January 19–20). Traffic Sign Detection based on HOG and PHOG using Binary SVM and k-NN. Proceedings of the 2016 3rd International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE), Semarang, Indonesia.10.1109/ICITACEE.2016.7892463 ↗
26. Image Recognition System using Geometric Matching and Contour Detection
27. A Kinematic Notation for Lower-pair Mechanisms Based on Matrices10.1115/1.4011045 ↗
28. Manigpan, S., Kiattisin, S., and Leelasantitham, A. (2010, January 27–30). A simulation of 6R Industrial Articulated Robot Arm using Backpropagation Neural Network. Proceedings of the ICCAS 2010, Gyeonggi-do, South Korea.10.1109/ICCAS.2010.5670125 ↗
29. Coordinated Motion Control of Robot Arms based on the Virtual Internal Model10.1109/70.127241 ↗
30. Önder Fractional Fuzzy Adaptive Sliding-Mode Control of a 2-DOF Direct-Drive Robot Arm10.1109/TSMCB.2008.928227 ↗
31. Bin, W.A., Daud, W., Faizura, W., Adly, M.A., Elamvazuthi, I., and Begam, M. (2010, January 15–17). Kinematic Modeling of Humanoid Arm. Proceedings of the 2010 International Conference on Intelligent and Advanced Systems, Manila, Philippines.
32. Wang, W.J., Huang, C.H., Lai, I.H., and Chen, H.C. (2010, January 18–21). A Robot Arm for Pushing Elevator Buttons. Proceedings of the SICE Annual Conference, Taipei, Taiwan.
33. Chang, W.C. (2010, January 18–21). Hybrid Fuzzy Control of an Eye-to-Hand Robotic Manipulator for Autonomous Assembly Tasks. Proceedings of the SICE Annual Conference, Taipei, Taiwan.
34. Collision avoidance using potential fields10.1108/01439919810240216 ↗
35. (2016, January 11). Histogram of Oriented Gradients. Available online: https://en.wikipedia.org/wiki/Histogram_of_oriented_gradients.
36. (2009, November 10). Support Vector Machine. Available online: http://en.wikipedia.org/wiki/Support_vector_machine.
37. Support vector clustering
38. Application of the LM-HLP Neural Network to Automatic Smartphone Test System