journal article·2024Open access

Investigation of Inside-Out Tracking Methods for Six Degrees of Freedom Pose Estimation of a Smartphone in Augmented Reality

Chanho Park, Takefumi Ogawa

Computers, Materials & Continua· Volume 79· Issue 2

Abstract

Six degrees of freedom (6DoF) input interfaces are essential for manipulating virtual objects through translation or rotation in three-dimensional (3D) space. A traditional outside-in tracking controller requires the installation of expensive hardware in advance. While inside-out tracking controllers have been proposed, they often suffer from limitations such as interaction limited to the tracking range of the sensor (e.g., a sensor on the head-mounted display (HMD)) or the need for pose value modification to function as an input interface (e.g., a sensor on the controller). This study investigates 6DoF pose estimation methods without restricting the tracking range, using a smartphone as a controller in augmented reality (AR) environments. Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach. In addition, seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor, the tracking method (e.g., marker recognition, visual-inertial odometry (VIO)), and whether modification of the initial pose is necessary. Through two experiments (discrete and continuous data), the performance of the algorithms was evaluated. The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose. Furthermore, the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Robotics and Sensor-Based Localization · Augmented Reality Applications · 3D Surveying and Cultural Heritage

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (31)

Show source-provided references
1. HTC, “HTC VIVE,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://www.vive.com
2. Microsoft, “MS HoloLens 2,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://www.microsoft.com/en-us/hololens/hardware
3. Meta, “Meta Quest 2,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://www.meta.com/us/en/quest/products/quest-2/
4. Google, “ARCore,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://developers.google.com/ar
5. Apple, “ARkit,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://developer.apple.com/augmented-reality/arkit
6. Google, “Google Cardboard,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://arvr.google.com/cardboard
7. Pocket6: A 6DoF controller based on a simple smartphone application
8. Insideout tracking controller for VR/AR HMD using image recognition with smartphones
9. Pair: Phone as an augmented immersive reality controller
10. OptiTrack, “Motion Capture Cameras,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://optitrack.com
11. Consumed endurance: A metric to quantify arm fatigue of mid-air interactions
12. Virtual object manipulation using a mobile phone
13. Dual phone AR: Exploring the use of phones as controllers for mobile augmented reality
14. TrackCap: Enabling smartphones for 3D interaction on mobile head-mounted displays
15. Combining multi-touch input and device movement for 3d manipulations in mobile augmented reality environments
16. Ultraleap, “Leap motion,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://www.ultraleap.com/product/leap-motion-controller
17. HandPoseMenu: Hand posture-based virtual menus for changing interaction mode in 3D space
18. LOP-cursor: Fast and precise interaction with tiled displays using one hand and levels of precision
19. Design and evaluation of a handheld-based 3D user interface for collaborative object manipulation
20. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM10.1109/TRO.2021.3075644 ↗
21. Strategy for creating AR applications in static and dynamic environments using SLAM- and marker detector-based tracking
22. AR Room: Real-time framework of camera location and interaction for augmented reality services
23. A SLAM-based 6DoF controller with smooth auto-calibration for virtual reality10.1007/s00371-022-02530-1 ↗
24. HandyCast: Phone-based bimanual input for virtual reality in mobile and space-constrained settings via pose-and-touch transfer
25. Unity 3D, “UNET 2019.4,” 2019. Accessed: Mar. 28, 2024. [Online]. Available: https://docs.unity3d.com/2019.4/Documentation/Manual/UNet.html
26. Bags of binary words for fast place recognition in image sequences10.1109/TRO.2012.2197158 ↗
27. Google, “Arcore cloud anchor,” 2024 Accessed: Mar. 28, 2024. [Online]. Available: https://developers.google.com/ar/develop/cloud-anchors
28. Generation of fiducial marker dictionaries using mixed integer linear programming10.1016/j.patcog.2015.09.023 ↗
29. Google, “Arcore recording and playback introduction,” 2024. Accessed: Mar. 28, 2024. [Online]. Available: https://developers.google.com/ar/develop/recording-and-playback
30. A benchmark for the evaluation of RGB-D SLAM systems
31. A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry