Enhancing Telexistence Control Through Assistive Manipulation and Haptic Feedback
Source abstract
Abstract
The COVID-19 pandemic brought telepresence systems into the spotlight, yet manually controlling remote robots often proves ineffective for handling complex manipulation tasks. To tackle this issue, we present a machine learning-based assistive manipulation approach. This method identifies target objects and computes an inverse kinematic solution for grasping them. The system integrates the generated solution with the user’s arm movements across varying inverse kinematic (IK) fusion levels. Given the importance of maintaining a sense of body ownership over the remote robot, we examine how haptic feedback and assistive functions influence ownership perception and task performance. Our findings indicate that incorporating assistance and haptic feedback significantly enhances the control of the robotic arm in telepresence environments, leading to improved precision and shorter task completion times. This research underscores the advantages of assistive manipulation techniques and haptic feedback in advancing telepresence technology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Teleoperation and Haptic Systems · Virtual Reality Applications and Impacts · Tactile and Sensory Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Qatar University, Doha 2713, Qatar
Department of Computer Science and Engineering, Waseda University, Tokyo 169-8555, Japan
School of Computing, Telkom University, Bandung 40257, Indonesia
Funding
Qatar University Marubeni-Qatar Japan Research Collaboration · KINDI Computing Research Center at Qatar University · Japan’s Ministry of Education, Culture, Sports, Science, and Technology
References (38)
Show source-provided references
1. Isabet, B., Pino, M., Lewis, M., Benveniste, S., and Rigaud, A.S. (2021). Social telepresence robots: A narrative review of experiments involving older adults before and during the COVID-19 pandemic. Int. J. Environ. Res. Public Health, 18.10.3390/ijerph18073597 ↗
2. Jain, A., Sharma, A., Wang, J., and Ram, M. (2021). Use of AI, Robotics, and Modern Tools to Fight COVID-19, CRC Press.10.1201/9781003339960 ↗
3. Saraiji, M.Y., Sasaki, T., Matsumura, R., Minamizawa, K., and Inami, M. (2018, January 12–16). Fusion: Full body surrogacy for collaborative communication. Proceedings of the ACM SIGGRAPH 2018 Emerging Technologies, Vancouver, BC, Canada.10.1145/3214907.3214912 ↗
4. Tachi, S., Minamizawa, K., Furukawa, M., and Fernando, C.L. (2012, January 7–12). Telexistence—From 1980 to 2012. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.
5. Body ownership and embodiment: Vestibular and multisensory mechanisms10.1016/j.neucli.2007.12.006 ↗
6. Whose Touch is This?: Understanding the Agency Trade-Off between User-Driven Touch vs. Computer-Driven Touch
7. Iskandar, A., Al-sada, M., Miyake, T., Saraiji, Y., Halabi, O., and Nakajima, T. (2022). Piton: Investigating the Controllability of a Wearable Telexistence Robot. Sensors, 22.10.3390/s22218574 ↗
8. Saraiji, M.Y. (2017). Embodied-Driven Design: A Framework to Configure Body Representation & Mapping. [Ph.D. Thesis, Keio University].
9. Eltanbouly, S., Abdel-Ghani, M., Helmy, H., Iskandar, A., Al-Sada, M., Nakajima, T., and Halabi, O. Assistive Telexistence System Using Motion Blending. Proceedings of the 2022 International Conference on Cyberworlds (CW), Kanazawa, Japan, 27–29 September 2022.10.1109/CW55638.2022.00023 ↗
10. Cho, K., Ko, K., Shim, H., and Jang, I. (July, January 28). Development of VR visualization system including deep learning architecture for improving teleoperability. Proceedings of the 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Republic of Korea.10.1109/URAI.2017.7992776 ↗
11. Hertkorn, K., Weber, B., Kremer, P., Roa, M.A., and Borst, C. (2013, January 15–17). Assistance for telepresence using online grasp planning. Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Atlanta, GA, USA.10.1109/HUMANOIDS.2013.7030021 ↗
12. Kamali, K., Bonev, I.A., and Desrosiers, C. (2020, January 13–15). Real-time Motion Planning for Robotic Teleoperation Using Dynamic-goal Deep Reinforcement Learning. Proceedings of the 2020 17th Conference on Computer and Robot Vision (CRV), Ottawa, ON, Canada.10.1109/CRV50864.2020.00032 ↗
13. Zhang, T., McCarthy, Z., Jowl, O., Lee, D., Chen, X., Goldberg, K., and Abbeel, P. (2018, January 21–25). Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation. Proceedings of the IEEE International Conference on Robotics and Automation, Brisbane, QLD, Australia.10.1109/ICRA.2018.8461249 ↗
14. Stoyanov, T., Krug, R., Kiselev, A., Sun, D., and Loutfi, A. (2018, January 1–5). Assisted Telemanipulation: A Stack-Of-Tasks Approach to Remote Manipulator Control. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Madrid, Spain.10.1109/IROS.2018.8594457 ↗
15. Dragan, A.D., and Srinivasa, S.S. (2013). Formalizing assistive teleoperation. Robotics: Science and Systems, MIT Press.10.15607/RSS.2012.VIII.010 ↗
16. Stanton, C., Bogdanovych, A., and Ratanasena, E. (2012, January 3–5). Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning. Proceedings of the Australasian Conference on Robotics and Automation, ACRA, Wellington, New Zealand.
17. A Cost-Effective Immersive Telexistence Platform for Generic Telemanipulation Tasks
18. Design and Implementation of Haptic Virtual Fixtures for Preoperative Surgical Planning10.1016/j.displa.2018.07.004 ↗
19. Sato, K., Minamizawa, K., Kawakami, N., and Tachi, S. (2007, January 5–9). Haptic telexistence. Proceedings of the SIGGRAPH’07: ACM SIGGRAPH 2007 Emerging Technologies, San Diego, CA, USA.10.1145/1278280.1278291 ↗
20. On the Decoding of Shoulder Joint Intent of Motion from Transient EMG: Feature Evaluation and Classification10.1109/TMRB.2023.3320260 ↗
21. Robotic telemanipulation with EMG-driven strategy-assisted shared control method10.1007/s11431-024-2779-8 ↗
22. Inoue, Y., Kato, F., Saraiji, M.Y., Fernando, C.L., and Tachi, S. (2017, January 18–22). Observation of mirror reflection and voluntary self-touch enhance self-recognition for a telexistence robot. Proceedings of the IEEE Virtual Reality, Los Angeles, CA, USA.10.1109/VR.2017.7892318 ↗
23. HTC Corporation (2025, January 01). VIVE. Available online: https://www.vive.com/.
24. Stereolabs Inc (2025, January 01). ZED Mini. Available online: https://www.stereolabs.com/zed-mini/.
25. (2025, January 01). GStreamer. Available online: https://gstreamer.freedesktop.org/.
26. Unity (2025, January 01). Unity Game Engine Web Site. Computer Software., Available online: https://unity.com/.
27. Urbani, J., Al-Sada, M., Nakajima, T., and Höglund, T. (2018, January 28–31). Exploring augmented reality interaction for everyday multipurpose wearable robots. Proceedings of the 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Hakodate, Japan.10.1109/RTCSA.2018.00033 ↗
28. Al-Sada, M., Höglund, T., Khamis, M., Urbani, J., and Nakajima, T. (2019, January 11–12). Orochi: Investigating Requirements and Expectations for Multipurpose Daily Used Supernumerary Robotic Limbs. Proceedings of the 10th Augmented Human International Conference 2019—AH2019, Reims, France.10.1145/3311823.3311850 ↗
29. Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation10.1109/TEVC.2018.2867601 ↗
30. ROBOTIS (2025, January 01). OpenManipulator-X (RM-X52-TNM). Available online: https://www.robotis.us/openmanipulator-x-rm-x52-tnm/.
31. ROBOTIS (2025, January 01). OpenManipulatorX. Available online: https://emanual.robotis.com/docs/en/platform/openmanipulator_x/specification/#hardware-specification.
32. Pololu Corporation (2025, January 01). Mini Maestro. Available online: https://www.pololu.com/product/1354.
33. Roboflow, I. (2025, January 01). YOLOv4-Tiny. Available online: https://models.roboflow.com/object-detection/yolov4-tiny-darknet.
34. Luxonis, I. (2025, January 01). OAK-D Camera. Available online: https://docs.luxonis.com/projects/hardware/en/latest/pages/BW1098OAK.html.
35. Quesnel, D., and Riecke, B.E. (2018). Are you awed yet? How virtual reality gives us awe and goose bumps. Front. Psychol., 9.10.3389/fpsyg.2018.02158 ↗
36. Roth, D., Lugrin, J.L., Latoschik, M.E., and Huber, S. (2017, January 6–11). Alpha IVBO—Construction of a scale to measure the illusion of virtual body ownership. Proceedings of the CHI EA’17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, Denver, CO, USA.10.1145/3027063.3053272 ↗
37. Contrast Coding: A Refinement of ANOVA in Behavioral Analysis
38. Are Multiple Contrast Tests Superior to the ANOVA?10.1515/ijb-2012-0020 ↗