Hierarchical Vision–Language Fusion with Structural Constraint Reasoning for Robust Multi-Jurisdiction License Plate Recognition
Source abstract
Abstract
Automatic License Plate Recognition (ALPR) in unconstrained traffic environments requires simultaneously addressing two fundamental challenges: reliable localization of small and degraded license plates and accurate decoding of visually ambiguous character sequences. This paper presents a hierarchical multi-stage framework that combines deep-learning-based detection, geometric normalization, dual-channel recognition, and structured post-correction to improve recognition robustness under diverse real-world conditions. A systematic ablation study involving five configurations (A0–A4) demonstrates the effectiveness of the proposed architecture across three benchmark datasets. On the UC3M-LP dataset, exact-match accuracy increases from 45.2% to 88.3%, while achieving 91.6% partial accuracy and a zero detection-miss rate. The framework further attains 95% exact-match accuracy on controlled European license plate crops and 93% on a large-scale custom dataset. In addition, we identify systematic evaluation artifacts in partially annotated benchmarks, showing that truncated ground-truth labels can underestimate genuine character-level improvements. The proposed framework supports multiple license plate formats through a configurable structural template library, and preliminary experiments on a small Arabic-script subset suggest potential extensibility without full model retraining. To ensure full reproducibility, all source code and evaluation resources are publicly released.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Vehicle License Plate Recognition · Advanced Neural Network Applications · IoT and GPS-based Vehicle Safety Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Laboratory of Electronics and Image Processing (LETI), Faculty of Electronics and Computer Science, Houari Boumediene University of Science and Technology (USTHB), Algiers 16111, Algeria
Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
Faculty of Sciences of Gabès, Cité Erriadh, Zrig, Gabès 6072, Tunisia
Research Team in Intelligent Machines (RTIM), University of Gabès, Gabès 6072, Tunisia
Funding
Princess Nourah bint Abdulrahman University
References (48)
Show source-provided references
1. License plate recognition from still images and video sequences: A survey10.1109/TITS.2008.922938 ↗
2. Automatic license plate recognition (ALPR): A state-of-the-art review10.1109/TCSVT.2012.2203741 ↗
3. Masood, S.Z., Shu, G., Dehghan, A., and Ortiz, E.G. (2017). License plate detection and recognition using deeply learned convolutional neural networks. arXiv.
4. Automatic license plate recognition10.1109/TITS.2004.825086 ↗
5. Trajectory data mining: An overview10.1145/2743025 ↗
6. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016, IEEE.10.1109/CVPR.2016.91 ↗
7. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv.
8. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 18–22 June 2023, IEEE.10.1109/CVPR52729.2023.00721 ↗
9. Jocher, G., Chaurasia, A., and Qiu, J. (2025, January 01). Ultralytics YOLOv8. Available online: https://github.com/ultralytics/ultralytics.
10. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017, IEEE.10.1109/ICCV.2017.324 ↗
11. An overview of the Tesseract OCR engine
12. Laroca, R., Severo, E., Zanlorensi, L.A., Oliveira, L.S., Gonçalves, G.R., Schwartz, W.R., and Menotti, D. (2018). A robust real-time automatic license plate recognition based on the YOLO detector. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018, IEEE.10.1109/IJCNN.2018.8489629 ↗
13. Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015). Spatial transformer networks. Advances in Neural Information Processing Systems, MIT Press.
14. Qiao, Y., Zhou, Z., Yang, D., Zhou, Y., and Wang, W. (2020). SEED: Semantics enhanced encoder-decoder framework for scene text recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 14–19 June 2020, IEEE.10.1109/CVPR42600.2020.01354 ↗
15. Selmi, Z., Halima, M.B., and Alimi, A.M. (2017). Deep learning system for automatic license plate detection and recognition. Proceedings of the 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan, 9–15 November 2017, IEEE.10.1109/ICDAR.2017.187 ↗
16. Real-time license plate detection and recognition using deep convolutional neural networks10.1016/j.jvcir.2020.102773 ↗
17. Montazzolli, S., and Jung, C. (2017). Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. Proceedings of the 30th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI), Rio de Janeiro, Brazil, 17–20 October 2017, IEEE.10.1109/SIBGRAPI.2017.14 ↗
18. An end-to-end trainable neural network for image-based sequence recognition10.1109/TPAMI.2016.2646371 ↗
19. Baek, Y., Lee, B., Han, D., Yun, S., and Lee, H. (2019). Character region awareness for text detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019, IEEE.10.1109/CVPR.2019.00959 ↗
20. TextBoxes++: A single-shot oriented scene text detector10.1109/TIP.2018.2825107 ↗
21. Decoupled attention network for text recognition
22. Multinational license plate recognition using generalized character sequence detection10.1109/ACCESS.2020.2974973 ↗
23. Robust license plate recognition under challenging conditions using deep learning
24. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Virtual, 18–24 July 2021, PMLR.
25. Li, J., Li, D., Xiong, C., and Hoi, S. (2022). BLIP: Bootstrapping language-image pre-training. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA, 17–23 July 2022, PMLR.
26. Liu, H., Li, C., Wu, Q., and Lee, Y.J. (2023). Visual instruction tuning. Advances in Neural Information Processing Systems, MIT Press.10.52202/075280-1516 ↗
27. Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., and Reynolds, M. (2022). Flamingo: A visual language model for few-shot learning. Advances in Neural Information Processing Systems, MIT Press.10.52202/068431-1723 ↗
28. Sivakoti, V., Nakka, R., and Bhatt, P. (2026). Neural Sentinel: Unified vision language model for license plate recognition. arXiv.
29. Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. (2023). MiniGPT-4: Enhancing vision-language understanding with advanced large language models. arXiv.
30. Hmimou, A., Medromi, H., and Sayouti, A. (2025). Automatic license plate recognition using YOLO and small language models. Proceedings of the IEEE International Conference on Electronics, Circuits and Systems (ICECS), Thessaloniki, Greece, 8–11 November 2026, IEEE.
31. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, MIT Press.
32. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and efficient foundation language models. arXiv.
33. OpenAI (2023). GPT-4 technical report. arXiv.
34. Diatom autofocusing in brightfield microscopy
35. Analysis of focus measure operators for shape-from-focus10.1016/j.patcog.2012.11.011 ↗
36. Adaptive histogram equalization and its variations10.1016/S0734-189X(87)80186-X ↗
37. Tomasi, C., and Manduchi, R. (1998). Bilateral filtering for gray and color images. Proceedings of the International Conference on Computer Vision (ICCV), Bombay, India, 4–7 January 1998, IEEE.10.1109/ICCV.1998.710815 ↗
38. A non-local algorithm for image denoising10.1109/CVPR.2005.38 ↗
39. Benchmark for license plate character segmentation10.1117/1.JEI.25.5.053034 ↗
40. Bodla, N., Singh, B., Chellappa, R., and Davis, L.S. (2017). Soft-NMS: Improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017, IEEE.10.1109/ICCV.2017.593 ↗
41. Binary codes capable of correcting deletions, insertions, and reversals
42. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
43. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, MIT Press.
44. Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., and Wang, X. (2022). ByteTrack: Multi-object tracking by associating every detection box. Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel, 23–27 October 2022, Springer.10.1007/978-3-031-20047-2_1 ↗
45. Finn, C., Abbeel, P., and Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International Conference on Machine Learning (ICML), Sydney, NSW, Australia, 6–11 August 2017, PMLR.
46. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK, 23–28 August 2020, Springer.10.1007/978-3-030-58452-8_13 ↗
47. Goodfellow, I.J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015, OpenReview.
48. McMahan, H.B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B.A. (2017). Communication-efficient learning of deep networks from decentralized data. Proceedings of the International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 20–22 April 2017, PMLR.