Connecting the Brain with Augmented Reality: A Systematic Review of BCI-AR Systems
Source abstract
Abstract
The increasing integration of brain–computer interfaces (BCIs) with augmented reality (AR) presents new possibilities for immersive and interactive environments, particularly through the use of head-mounted displays (HMDs). Despite the growing interest, a comprehensive understanding of BCI-AR systems is still emerging. This systematic review aims to synthesize existing research on the use of BCIs for controlling AR environments via HMDs, highlighting the technological advancements and challenges in this domain. An extensive search across electronic databases, including IEEEXplore, PubMed, and Scopus, was conducted following the PRISMA guidelines, resulting in 41 studies eligible for analysis. This review identifies key areas for future research, potential limitations, and offers insights into the evolving trends in BCI-AR systems, contributing to the development of more robust and user-friendly applications.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: EEG and Brain-Computer Interfaces · Gaze Tracking and Assistive Technology · Functional Brain Connectivity Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Computer Engineering, University of Western Macedonia, Campus ZEP Kozani, 50100 Kozani, Greece
Funding
“AGROTOUR–New Technologies and Innovative Approaches to Agri-Food and Tourism to Boost Regional Excellence in Western Macedonia” · “Competitiveness, Entrepreneurship and Innovation”
References (54)
Show source-provided references
1. Graz brain-computer interface II: Towards communication between humans and computers based on online classification of three different EEG patterns10.1007/BF02520010 ↗
2. Brain–computer interfaces for communication and control10.1016/S1388-2457(02)00057-3 ↗
3. Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain–computer interface10.1088/1741-2560/12/4/046008 ↗
4. Sensor modalities for brain-computer interface technology: A comprehensive literature review10.1093/neuros/nyz286 ↗
5. Waldert, S. (2016). Invasive vs. non-invasive neuronal signals for brain-machine interfaces: Will one prevail?. Front. Neurosci., 10.10.3389/fnins.2016.00295 ↗
6. Motor imagery and direct brain-computer communication10.1109/5.939829 ↗
7. Towards passive brain–computer interfaces: Applying brain–computer interface technology to human–machine systems in general10.1088/1741-2560/8/2/025005 ↗
8. Lotte, F., Faller, J., Guger, C., Renard, Y., Pfurtscheller, G., Lécuyer, A., and Leeb, R. (2013). Combining BCI with virtual reality: Towards new applications and improved BCI. Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications, Springer.10.1007/978-3-642-29746-5_10 ↗
9. A review on Virtual Reality and Augmented Reality use-cases of Brain Computer Interface based applications for smart cities10.1016/j.micpro.2021.104392 ↗
10. Wearable Brain-Computer Interfaces based on Steady-State Visually Evoked Potentials and Augmented Reality: A Review10.1109/JSEN.2023.3287983 ↗
11. EEG-Based Brain-Computer Interactions in Immersive Virtual and Augmented Reality: A Systematic Review10.1145/3593226 ↗
12. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement10.7326/0003-4819-151-4-200908180-00135 ↗
13. (2023, June 01). Rayyan. Intelligent Systematic Review—Rayyan. Available online: https://www.rayyan.ai/.
14. Putze, F., Weiß, D., Vortmann, L.M., and Schultz, T. (2019, January 6–9). Augmented reality interface for smart home control using SSVEP-BCI and eye gaze. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.10.1109/SMC.2019.8914390 ↗
15. Development of an online home appliance control system using augmented reality and an SSVEP-based brain–computer interface10.1109/ACCESS.2019.2952613 ↗
16. Kim, J.W., Kim, M.N., Kang, D.H., Ahn, M.H., Kim, H.S., and Min, B.K. (2019, January 18–20). An online top-down SSVEP-BMI for augmented reality. Proceedings of the IEEE 2019 7th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Republic of Korea.10.1109/IWW-BCI.2019.8737348 ↗
17. Towards BCI-based interfaces for augmented reality: Feasibility, design and evaluation
18. Angrisani, L., Arpaia, P., Moccaldi, N., and Esposito, A. (2018, January 10–13). Wearable augmented reality and brain computer interface to improve human-robot interactions in smart industry: A feasibility study for SSVEP signals. Proceedings of the IEEE 2018 IEEE 4th International Forum on Research and Technology for Society and Industry (RTSI), Palermo, Italy.10.1109/RTSI.2018.8548517 ↗
19. SSVEP stimulus layout effect on accuracy of brain-computer interfaces in augmented reality glasses10.1109/ACCESS.2019.2963442 ↗
20. Liu, P., Ke, Y., Du, J., Liu, W., Kong, L., Wang, N., An, X., and Ming, D. (2019, January 23–27). An SSVEP-BCI in augmented reality. Proceedings of the IEEE 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany.10.1109/EMBC.2019.8857859 ↗
21. Kerous, B., and Liarokapis, F. (2017, January 9–13). BrainChat-A collaborative augmented reality brain interface for message communication. Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), Nantes, France.10.1109/ISMAR-Adjunct.2017.91 ↗
22. Combination of augmented reality based brain-computer interface and computer vision for high-level control of a robotic arm10.1109/TNSRE.2020.3038209 ↗
23. A wearable brain–computer interface instrument for augmented reality-based inspection in industry 4.010.1109/TIM.2019.2914712 ↗
24. An online SSVEP-BCI system in an optical see-through augmented reality environment10.1088/1741-2552/ab4dc6 ↗
25. Improving AR-SSVEP Recognition Accuracy Under High Ambient Brightness Through Iterative Learning10.1109/TNSRE.2023.3260842 ↗
26. Heo, D., Kim, M., Kim, J., Choi, Y.J., and Kim, S.P. (2022, January 21–23). The Uses of Brain-Computer Interface in Different Postures to Application in Real Life. Proceedings of the IEEE 2022 10th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Republic of Korea.10.1109/BCI53720.2022.9734957 ↗
27. Study on robot grasping system of SSVEP-BCI based on augmented reality stimulus10.26599/TST.2021.9010085 ↗
28. Jang, H., Park, S., Woo, J., Ha, J., and Kim, L. (2023, January 20–22). Authentication System Based on Event-related Potentials Using AR Glasses. Proceedings of the IEEE 2023 11th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Republic of Korea.10.1109/BCI57258.2023.10078487 ↗
29. Brain-controlled, AR-based Home automation system using SSVEP-based brain-computer interface and EOG-based eye tracker: A feasibility study for the elderly end User10.1109/TNSRE.2022.3228124 ↗
30. Brain-computer interface integrated with augmented reality for human-robot interaction10.1109/TCDS.2022.3194603 ↗
31. Angrisani, L., Apicella, A., Arpaia, P., De Benedetto, E., Donato, N., Duraccio, L., Giugliano, S., and Prevete, R. (2022, January 16–19). A ML-based Approach to Enhance Metrological Performance of Wearable Brain-Computer Interfaces. Proceedings of the 2022 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Ottawa, ON, Canada.10.1109/I2MTC48687.2022.9806518 ↗
32. Enhancement of SSVEPs classification in BCI-based wearable instrumentation through machine learning techniques10.1109/JSEN.2022.3161743 ↗
33. Augmented reality driven steady-state visual evoked potentials for wheelchair navigation10.1109/TNSRE.2022.3215695 ↗
34. Huang, X., Mak, J., Wears, A., Price, R.B., Akcakaya, M., Ostadabbas, S., and Woody, M.L. (2022, January 11–15). Using Neurofeedback from Steady-State Visual Evoked Potentials to Target Affect-Biased Attention in Augmented Reality. Proceedings of the IEEE 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK.10.1109/EMBC48229.2022.9871982 ↗
35. Apicella, A., Arpaia, P., Cataldo, A., De Benedetto, E., Donato, N., Duraccio, L., Giugliano, S., and Prevete, R. (2022, January 22–24). Adoption of Machine Learning Techniques to Enhance Classification Performance in Reactive Brain-Computer Interfaces. Proceedings of the 2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Messina, Italy.10.1109/MeMeA54994.2022.9856441 ↗
36. Augmented Reality Brain–Computer Interface with Spatial Awareness10.1002/tee.23692 ↗
37. A novel brain-controlled wheelchair combined with computer vision and augmented reality10.1186/s12938-022-01020-8 ↗
38. He, C., Du, Y., and Zhao, X. (2022). A separable convolutional neural network-based fast recognition method for AR-P300. Front. Hum. Neurosci., 16.10.3389/fnhum.2022.986928 ↗
39. Du, Y., and Zhao, X. (2022). Visual stimulus color effect on SSVEP-BCI in augmented reality. Biomed. Signal Process. Control, 78.10.1016/j.bspc.2022.103906 ↗
40. Performance enhancement of wearable instrumentation for AR-based SSVEP BCI10.1016/j.measurement.2022.111188 ↗
41. The effect of stimulus number on the recognition accuracy and information transfer rate of SSVEP–BCI in augmented reality10.1088/1741-2552/ac6ae5 ↗
42. Horii, S., Nakauchi, S., and Kitazaki, M. (2015, January 23–27). AR-SSVEP for brain-machine interface: Estimating user’s gaze in head-mounted display with USB camera. Proceedings of the 2015 IEEE Virtual Reality (VR), Arles, France.10.1109/VR.2015.7223361 ↗
43. Supporting Human–Robot Interaction by Projected Augmented Reality and a Brain Interface10.1109/THMS.2024.3414208 ↗
44. A novel brain-controlled prosthetic hand method integrating AR-SSVEP augmentation, asynchronous control, and machine vision assistance10.1016/j.heliyon.2024.e26521 ↗
45. Control of Electric Wheelchair by Brain-Computer Interface Using Mixed Reality and Virtual Sound Source10.1002/tee.24050 ↗
46. Olivieri, E., Barresi, G., and Mattos, L.S. (2015, January 25–29). BCI-based user training in surgical robotics. Proceedings of the IEEE 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.10.1109/EMBC.2015.7319495 ↗
47. Vortmann, L.M., Kroll, F., and Putze, F. (2019). EEG-based classification of internally-and externally-directed attention in an augmented reality paradigm. Front. Hum. Neurosci., 13.10.3389/fnhum.2019.00348 ↗
48. Vortmann, L.M., Schwenke, L., and Putze, F. (2021). Using brain activity patterns to differentiate real and virtual attended targets during augmented reality scenarios. Information, 12.10.3390/info12060226 ↗
49. Sanna, A., Manuri, F., Fiorenza, J., and De Pace, F. (2022). BARI: An Affordable Brain-Augmented Reality Interface to Support Human–Robot Collaboration in Assembly Tasks. Information, 13.10.3390/info13100460 ↗
50. De Massari, D., Pacheco, D., Malekshahi, R., Betella, A., Verschure, P.F., Birbaumer, N., and Caria, A. (2014). Fast mental states decoding in mixed reality. Front. Behav. Neurosci., 8.10.3389/fnbeh.2014.00415 ↗
51. Choi, J., and Jo, S. (2020, January 26–28). Application of hybrid Brain-Computer Interface with augmented reality on quadcopter control. Proceedings of the IEEE 2020 8th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Republic of Korea.10.1109/BCI48061.2020.9061659 ↗
52. Horie, R., Goto, K., and Ootsuka, Y. (2018, January 9–12). Game systems by using a brain computer interface and mixed reality. Proceedings of the 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), Nara, Japan.10.1109/GCCE.2018.8574845 ↗
53. A closed-loop brain-computer interface with augmented reality feedback for industrial human-robot collaboration10.1007/s00170-021-07937-z ↗
54. Sun, H., Yan, X., Yang, J., and Li, Q. (2024, January 12–14). Development of a brain-controlled robotic arm system for motor imagery based on MR visual guidance. Proceedings of the 2024 4th International Conference on Bioinformatics and Intelligent Computing, Dalian, China.10.1145/3665689.3665716 ↗