A Parallel Open-World Object Detection Framework with Uncertainty Mitigation for Campus Monitoring
Source abstract
Abstract
The recent advancements in artificial intelligence have brought about significant changes in education. In the context of intelligent campus development, target detection technology plays a pivotal role in applications such as campus environment monitoring and the facilitation of classroom behavior surveillance. However, traditional object detection methods face challenges in open and dynamic campus scenarios where unexpected objects and behaviors arise. Open-World Object Detection (OWOD) addresses this issue by enabling detectors to gradually learn and recognize unknown objects. Nevertheless, existing OWOD methods introduce two major uncertainties that limit the detection performance: the unknown discovery uncertainty from the manual generation of pseudo-labels for unknown objects and the known discrimination uncertainty from perturbations that unknown training introduces to the known class features. In this paper, we introduce a Parallel OWOD Framework with Uncertainty Mitigation to alleviate the unknown discovery uncertainty and the known discrimination uncertainty within the OWOD task. To address the unknown discovery uncertainty, we propose an objectness-driven discovery module to focus on capturing the generalized objectness shared among various known classes, driving the framework to discover more potential objects that are distinct from the background, including unknown objects. To mitigate the discrimination uncertainty, we decouple the learning processes for known and unknown classes through a parallel structure to reduce the mutual influence at the feature level and design a collaborative open-world classifier to achieve high-performance collaborative detection of both known and unknown classes. Our framework provides educators with a powerful tool for effective campus monitoring and classroom management. Experimental results on standard benchmarks demonstrate the framework’s superior performance compared to state-of-the-art methods, showcasing its transformative potential in intelligent educational environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · COVID-19 diagnosis using AI · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
State Key Lab of Software Development Environment, Beihang University, Beijing 100191, China
China Electronics Standardization Institute, Beijing 100007, China
School of Computer Science, Peking University, Beijing 100871, China
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
Institute of Artificial Intelligence, Beihang University, Beijing 100191, China
Beijing Institute of Control and Electronic Technology, Beijing 100038, China
Funding
National Natural Science Foundation of China · National Key R&D Program of China
References (57)
Show source-provided references
1. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
2. Arcucci, R., Zhu, J., Hu, S., and Guo, Y.K. (2021). Deep data assimilation: Integrating deep learning with data assimilation. Appl. Sci., 11.10.3390/app11031114 ↗
3. Li, F., He, F., Wang, F., Zhang, D., Xia, Y., and Li, X. (2020). A novel simplified convolutional neural network classification algorithm of motor imagery EEG signals based on deep learning. Appl. Sci., 10.10.3390/app10051605 ↗
4. Shieh, C.S., Lin, W.W., Nguyen, T.T., Chen, C.H., Horng, M.F., and Miu, D. (2021). Detection of unknown ddos attacks with deep learning and gaussian mixture model. Appl. Sci., 11.10.3390/app11115213 ↗
5. Chiu, M.T., Xu, X., Wei, Y., Huang, Z., Schwing, A.G., Brunner, R., Khachatrian, H., Karapetyan, H., Dozier, I., and Rose, G. (2020, January 13–19). Agriculture-vision: A large aerial image database for agricultural pattern analysis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00290 ↗
6. Ni, J., Chen, Y., Chen, Y., Zhu, J., Ali, D., and Cao, W. (2020). A survey on theories and applications for self-driving cars based on deep learning methods. Appl. Sci., 10.10.3390/app10082749 ↗
7. Rehman, A., Iqbal, M.A., Xing, H., and Ahmed, I. (2021). COVID-19 detection empowered with machine learning and deep learning techniques: A systematic review. Appl. Sci., 11.10.3390/app11083414 ↗
8. Zhou, H., Jiang, F., and Shen, R. (2018, January 14–16). Who are raising their hands? Hand-raiser seeking based on object detection and pose estimation. Proceedings of the Asian Conference on Machine Learning, PMLR, Beijing, China.
9. CBPH-Net: A Small Object Detector for Behavior Recognition in Classroom Scenarios
10. Liu, H., Yu, Y., Liu, S., and Wang, W. (2022). A Military Object Detection Model of UAV Reconnaissance Image and Feature Visualization. Appl. Sci., 12.10.3390/app122312236 ↗
11. Park, Y., and Shin, Y. (2023). Applying Object Detection and Embedding Techniques to One-Shot Class-Incremental Multi-Label Image Classification. Appl. Sci., 13.10.3390/app131810468 ↗
12. Miao, B., Chen, Z., Liu, H., and Zhang, A. (2023). A target re-identification method based on shot boundary object detection for single object tracking. Appl. Sci., 13.10.3390/app13116422 ↗
13. Analyzing students’ attention by gaze tracking and object detection in classroom teaching
14. Joseph, K., Khan, S., Khan, F.S., and Balasubramanian, V.N. (2021, January 20–25). Towards open world object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00577 ↗
15. Zhao, X., Liu, X., Shen, Y., Ma, Y., Qiao, Y., and Wang, D. (2022). Revisiting open world object detection. arXiv.10.1109/TCSVT.2023.3326279 ↗
16. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J.F.D.D. (2021;, January 3–7). Deformable transformers for end-to-end object detection. Proceedings of the 9th International Conference on Learning Representations, Virtual Event, Austria.
17. Gupta, A., Narayan, S., Joseph, K., Khan, S., Khan, F.S., and Shah, M. (2022, January 18–24). OW-DETR: Open-world detection transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00902 ↗
18. Chen, C., Seff, A., Kornhauser, A., and Xiao, J. (2015, January 7–13). Deepdriving: Learning affordance for direct perception in autonomous driving. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.312 ↗
19. Guo, S., Wang, S., Yang, Z., Wang, L., Zhang, H., Guo, P., Gao, Y., and Guo, J. (2022). A Review of Deep Learning-Based Visual Multi-Object Tracking Algorithms for Autonomous Driving. Appl. Sci., 12.10.3390/app122110741 ↗
20. Lee, Y., and Park, S. (2020). A deep learning-based perception algorithm using 3D lidar for autonomous driving: Simultaneous segmentation and detection network (ssadnet). Appl. Sci., 10.10.3390/app10134486 ↗
21. Medical image segmentation on GPUs—A comprehensive review10.1016/j.media.2014.10.012 ↗
22. Nagi, A.T., Awan, M.J., Mohammed, M.A., Mahmoud, A., Majumdar, A., and Thinnukool, O. (2022). Performance analysis for COVID-19 diagnosis using custom and state-of-the-art deep learning models. Appl. Sci., 12.10.3390/app12136364 ↗
23. Qureshi, S.A., Raza, S.E.A., Hussain, L., Malibari, A.A., Nour, M.K., Rehman, A.U., Al-Wesabi, F.N., and Hilal, A.M. (2022). Intelligent ultra-light deep learning model for multi-class brain tumor detection. Appl. Sci., 12.10.3390/app12083715 ↗
24. Peng, T., Liang, Y., Wu, W., Ren, J., Pengrui, Z., and Pu, Y. (2023, January 7–14). CLGT: A graph transformer for student performance prediction in collaborative learning. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i13.26893 ↗
25. Tsiakmaki, M., Kostopoulos, G., Kotsiantis, S., and Ragos, O. (2020). Transfer learning from deep neural networks for predicting student performance. Appl. Sci., 10.10.3390/app10062145 ↗
26. HELP-DKT: An interpretable cognitive model of how students learn programming based on deep knowledge tracing10.1038/s41598-022-07956-0 ↗
27. Hand-raising gesture detection in real classrooms using improved R-FCN10.1016/j.neucom.2019.05.031 ↗
28. Regionwise generative adversarial image inpainting for large missing areas10.1109/TCYB.2022.3194149 ↗
29. Hu, S., Ma, Y., Liu, X., Wei, Y., and Bai, S. (2021, January 2–9). Stratified rule-aware network for abstract visual reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i2.16248 ↗
30. Open set face recognition using transduction10.1109/TPAMI.2005.224 ↗
31. Heflin, B., Scheirer, W., and Boult, T.E. (2012, January 23–27). Detecting and classifying scars, marks, and tattoos found in the wild. Proceedings of the 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS), Arlington, VA, USA.10.1109/BTAS.2012.6374555 ↗
32. Pritsos, D.A., and Stamatatos, E. (2013, January 24–27). Open-set classification for automated genre identification. Proceedings of the European Conference on Information Retrieval, Moscow, Russia.10.1007/978-3-642-36973-5_18 ↗
33. Open set recognition for automatic target classification with rejection10.1109/TAES.2015.150027 ↗
34. Fei, G., and Liu, B. (2016, January 12–17). Breaking the closed world assumption in text classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.10.18653/v1/N16-1061 ↗
35. Vareto, R., Silva, S., Costa, F., and Schwartz, W.R. (2017, January 1–4). Towards open-set face recognition using hashing functions. Proceedings of the 2017 IEEE International Joint Conference on Biometrics (IJCB), Denver, CO, USA.10.1109/BTAS.2017.8272751 ↗
36. Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., and Yu, S.X. (2019, January 15–20). Large-scale long-tailed recognition in an open world. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00264 ↗
37. Bendale, A., and Boult, T.E. (July, January 26). Towards open set deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
38. Perera, P., Morariu, V.I., Jain, R., Manjunatha, V., Wigington, C., Ordonez, V., and Patel, V.M. (2020, January 13–19). Generative-discriminative feature representations for open-set recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01183 ↗
39. Yoshihashi, R., Shao, W., Kawakami, R., You, S., Iida, M., and Naemura, T. (2019, January 15–20). Classification-reconstruction learning for open-set recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00414 ↗
40. Dhamija, A., Gunther, M., Ventura, J., and Boult, T. (2020, January 1–5). The overlooked elephant of object detection: Open set. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093355 ↗
41. Miller, D., Nicholson, L., Dayoub, F., and Sünderhauf, N. (2018, January 21–25). Dropout sampling for robust object detection in open-set conditions. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.10.1109/ICRA.2018.8460700 ↗
42. Miller, D., Dayoub, F., Milford, M., and Sünderhauf, N. (2019, January 20–24). Evaluating merging strategies for sampling-based uncertainty techniques in object detection. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8793821 ↗
43. Hall, D., Dayoub, F., Skinner, J., Zhang, H., Miller, D., Corke, P., Carneiro, G., Angelova, A., and Sünderhauf, N. (2020, January 1–5). Probabilistic object detection: Definition and evaluation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093599 ↗
44. Han, J., Ren, Y., Ding, J., Pan, X., Yan, K., and Xia, G.S. (2022, January 18–24). Expanding Low-Density Latent Regions for Open-Set Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00937 ↗
45. Bendale, A., and Boult, T. (2015, January 7–12). Towards open world recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298799 ↗
46. Xu, H., Liu, B., Shu, L., and Yu, P. (2019, January 13–17). Open-world learning and application to product classification. Proceedings of the The World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313644 ↗
47. Willes, J., Harrison, J., Harakeh, A., Finn, C., Pavone, M., and Waslander, S. (2022). Bayesian embeddings for few-shot open world recognition. IEEE Trans. Pattern Anal. Mach. Intell., 1–16.10.1109/TPAMI.2022.3201541 ↗
48. Mancini, M., Naeem, M.F., Xian, Y., and Akata, Z. (2021, January 20–25). Open world compositional zero-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00518 ↗
49. Zhang, S., Li, Z., Yan, S., He, X., and Sun, J. (2021, January 20–25). Distribution alignment: A unified framework for long-tail visual recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00239 ↗
50. Generalizing from a few examples: A survey on few-shot learning
51. Xian, Y., Schiele, B., and Akata, Z. (2017, January 21–26). Zero-shot learning-the good, the bad and the ugly. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.328 ↗
52. Yang, S., Sun, P., Jiang, Y., Xia, X., Zhang, R., Yuan, Z., Wang, C., Luo, P., and Xu, M. (2021). Objects in semantic topology. arXiv.
53. Wu, Z., Lu, Y., Chen, X., Wu, Z., Kang, L., and Yu, J. (2022). UC-OWOD: Unknown-Classified Open World Object Detection. arXiv.10.1007/978-3-031-20080-9_12 ↗
54. Attention is all you need
55. The pascal visual object classes (voc) challenge10.1007/s11263-009-0275-4 ↗
56. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
57. Faster r-cnn: Towards real-time object detection with region proposal networks