CriSALAD: Robust Visual Place Recognition Using Cross-Image Information and Optimal Transport Aggregation
Source abstract
Abstract
Visual place recognition (VPR) is crucial for enabling autonomous agents to accurately localize themselves within a known environment. While existing methods leverage neural networks to enhance performance and robustness, they often suffer from the limited representation power of local feature extractors. To address this limitation, we propose CriSALAD, a novel VPR model that integrates visual foundation models (VFMs) and cross-image information to improve feature extraction robustness. Specifically, we adapt pre-trained VFMs for VPR by incorporating a parameter-efficient adapter inspired by Xception, ensuring effective task adaptation while preserving computational efficiency. Additionally, we employ the Sinkhorn Algorithm for Locally Aggregated Descriptors (SALAD) as a global descriptor to enhance place recognition accuracy. Furthermore, we introduce a transformer-like cross-image encoder that facilitates information sharing between neighboring images, thus enhancing feature representations. We evaluate CriSALAD on multiple publicly available place recognition datasets, achieving promising performance with a recall@1 of 89.3% on the Nordland dataset, while the closest rival achieves only 76.2%. CriSALAD outperforms both baseline models and advanced VFM-based VPR approaches.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image and Video Retrieval Techniques · Robotics and Sensor-Based Localization · Indoor and Outdoor Localization Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China
Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
Funding
Zhejiang Provincial Natural Science Foundation of China
References (49)
Show source-provided references
1. A survey on deep visual place recognition10.1109/ACCESS.2021.3054937 ↗
2. Schubert, S., Neubert, P., Garg, S., Milford, M., and Fischer, T. (2023). Visual place recognition: A tutorial. arXiv.10.1109/MRA.2023.3310859 ↗
3. Chen, Z., Jacobson, A., Sünderhauf, N., Upcroft, B., Liu, L., Shen, C., Reid, I., and Milford, M. (June, January 29). Deep learning features at scale for visual place recognition. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989366 ↗
4. Middelberg, S., Sattler, T., Untzelmann, O., and Kobbelt, L. (2014, January 6–12). Scalable 6-dof localization on mobile devices. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part II 13.10.1007/978-3-319-10605-2_18 ↗
5. Qian, J., Cheng, Y., Ying, R., and Liu, P. (2020). A Novel Indoor Localization Method Based on Image Retrieval and Dead Reckoning. Appl. Sci., 10.10.3390/app10113803 ↗
6. Peng, Z., Song, R., Yang, H., Li, Y., Lin, J., Xiao, Z., and Yan, B. (2024). Adaptive Feature Refinement and Weighted Similarity for Deep Loop Closure Detection in Appearance Variation. Appl. Sci., 14.10.3390/app14146276 ↗
7. Garg, S., Sünderhauf, N., and Milford, M. (2018, January 26–30). LoST? Appearance-Invariant Place Recognition for Opposite Viewpoints using Visual Semantics. Proceedings of the Robotics: Science and Systems XIV, Carnegie Mellon University, Pittsburgh, PA, USA.10.15607/RSS.2018.XIV.022 ↗
8. Ming, Y., Yang, X., and Calway, A. (October, January 27). Object-Augmented RGB-D SLAM for Wide-Disparity Relocalisation. Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.10.1109/IROS51168.2021.9636034 ↗
9. Berton, G., Trivigno, G., Caputo, B., and Masone, C. (2023, January 1–6). EigenPlaces: Training Viewpoint Robust Models for Visual Place Recognition. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01017 ↗
10. Ming, Y., Yang, X., Zhang, G., and Calway, A. (2022, January 23–27). CGiS-Net: Aggregating Colour, Geometry and Implicit Semantic Features for Indoor Place Recognition. Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.10.1109/IROS47612.2022.9981113 ↗
11. Ming, Y., Ma, J., Yang, X., Dai, W., Peng, Y., and Kong, W. (2024, January 14–19). AEGIS-Net: Attention-Guided Multi-Level Feature Aggregation for Indoor Place Recognition. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447578 ↗
12. Speeded-up robust features (SURF)10.1016/j.cviu.2007.09.014 ↗
13. Jégou, H., Douze, M., Schmid, C., and Pérez, P. (2010, January 13–18). Aggregating local descriptors into a compact image representation. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5540039 ↗
14. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6–13). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126544 ↗
15. Fast and incremental method for loop-closure detection using bags of visual words10.1109/TRO.2008.2004514 ↗
16. Aggregation of Deep Local Features using VLAD and Classification using R2 Forest10.1016/j.procs.2018.10.334 ↗
17. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv.
18. AnyLoc: Towards Universal Visual Place Recognition10.1109/LRA.2023.3343602 ↗
19. Lu, F., Lan, X., Zhang, L., Jiang, D., Wang, Y., and Yuan, C. (2024, January 16–22). CricaVPR: Cross-Image Correlation-Aware Representation Learning for Visual Place Recognition. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01587 ↗
20. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., and Sivic, J. (2016, January 27–30). NetVLAD: CNN architecture for weakly supervised place recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.572 ↗
21. Ali-Bey, A., Chaib-Draa, B., and Giguere, P. (2023, January 2–7). Mixvpr: Feature mixing for visual place recognition. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV56688.2023.00301 ↗
22. Fine-tuning CNN image retrieval with no human annotation
23. Izquierdo, S., and Civera, J. (2024, January 16–22). Optimal transport aggregation for visual place recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01672 ↗
24. Hausler, S., Garg, S., Xu, M., Milford, M., and Fischer, T. (2021, January 20–25). Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01392 ↗
25. Hardness-Aware Metric Learning With Cluster-Guided Attention for Visual Place Recognition10.1109/TCSVT.2024.3465520 ↗
26. VIPeR: Visual Incremental Place Recognition With Adaptive Mining and Continual Learning10.1109/LRA.2025.3539093 ↗
27. Yang, X., Ming, Y., Cui, Z., and Calway, A. (2022, January 23–27). FD-SLAM: 3-D Reconstruction Using Features and Dense Matching. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.10.1109/ICRA46639.2022.9812049 ↗
28. Oh, J., and Eoh, G. (2021). Variational Bayesian Approach to Condition-Invariant Feature Extraction for Visual Place Recognition. Appl. Sci., 11.10.3390/app11198976 ↗
29. Ye, M., and Tanaka, K. (2023). Visual Place Recognition of Robots via Global Features of Scan-Context Descriptors with Dictionary-Based Coding. Appl. Sci., 13.10.3390/app13159040 ↗
30. Chen, B., Song, X., Shen, H., and Lu, T. (2021). Hierarchical Visual Place Recognition Based on Semantic-Aggregation. Appl. Sci., 11.10.3390/app11209540 ↗
31. Berton, G., Masone, C., and Caputo, B. (2022, January 18–24). Rethinking Visual Geo-localization for Large-Scale Applications. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00483 ↗
32. Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., and El-Nouby, A. (2023). Dinov2: Learning robust visual features without supervision. arXiv.
33. Huang, G., Zhou, Y., Hu, X., Zhang, C., Zhao, L., and Gan, W. (2024). DINO-Mix enhancing visual place recognition with foundational vision model and feature mixing. Sci. Rep., 14.10.1038/s41598-024-73853-3 ↗
34. Lu, F., Zhang, L., Lan, X., Dong, S., Wang, Y., and Yuan, C. (2024, January 7–11). Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition. Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria.
35. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv.
36. He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., and Neubig, G. (2021). Towards a unified view of parameter-efficient transfer learning. arXiv.
37. Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019, January 9–15). Parameter-efficient transfer learning for NLP. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA.
38. Lester, B., Al-Rfou, R., and Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. arXiv.10.18653/v1/2021.emnlp-main.243 ↗
39. Adaptformer: Adapting vision transformers for scalable visual recognition
40. Xu, M., Zhang, Z., Wei, F., Hu, H., and Bai, X. (2023, January 17–24). Side adapter network for open-vocabulary semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00288 ↗
41. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
42. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is All you Need. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA.
43. GSV-Cities: Toward appropriate supervised visual place recognition10.1016/j.neucom.2022.09.127 ↗
44. Torii, A., Sivic, J., Pajdla, T., and Okutomi, M. (2013, January 23–28). Visual Place Recognition with Repetitive Structures. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.119 ↗
45. 24/7 Place Recognition by View Synthesis10.1109/TPAMI.2017.2667665 ↗
46. Olid, D., Fácil, J.M., and Civera, J. (2018). Single-View Place Recognition under Seasonal Changes. arXiv.
47. Warburg, F., Hauberg, S., López-Antequera, M., Gargallo, P., Kuang, Y., and Civera, J. (2020, January 13–19). Mapillary Street-Level Sequences: A Dataset for Lifelong Place Recognition. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00270 ↗
48. Loshchilov, I., and Hutter, F. (2019, January 6–9). Decoupled Weight Decay Regularization. Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA.
49. Wang, R., Shen, Y., Zuo, W., Zhou, S., and Zheng, N. (2022, January 18–24). TransVPR: Transformer-Based Place Recognition with Multi-Level Attention Aggregation. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01328 ↗