A Novel Discriminating and Relative Global Spatial Image Representation with Applications in CBIR
Source abstract
Abstract
The requirement for effective image search, which motivates the use of Content-Based Image Retrieval (CBIR) and the search of similar multimedia contents on the basis of user query, remains an open research problem for computer vision applications. The application domains for Bag of Visual Words (BoVW) based image representations are object recognition, image classification and content-based image analysis. Interest point detectors are quantized in the feature space and the final histogram or image signature do not retain any detail about co-occurrences of features in the 2D image space. This spatial information is crucial, as it adversely affects the performance of an image classification-based model. The most notable contribution in this context is Spatial Pyramid Matching (SPM), which captures the absolute spatial distribution of visual words. However, SPM is sensitive to image transformations such as rotation, flipping and translation. When images are not well-aligned, SPM may lose its discriminative power. This paper introduces a novel approach to encoding the relative spatial information for histogram-based representation of the BoVW model. This is established by computing the global geometric relationship between pairs of identical visual words with respect to the centroid of an image. The proposed research is evaluated by using five different datasets. Comprehensive experiments demonstrate the robustness of the proposed image representation as compared to the state-of-the-art methods in terms of precision and recall values.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques · Remote-Sensing Image Classification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, National Textile University, Faisalabad 38000, Pakistan
Department of Computer Science, Government College University, Faisalabad 38000, Pakistan
Department of Software Engineering, Mirpur University of Science & Technology, Mirpur AJK 10250, Pakistan
Computer Aided Automation, Computer Vision Lab, Vienna University of Technology, A-1040 Vienna, Austria
Department of Mathematics, Government College University, Faisalabad 38000, Pakistan
Department of Electrical Engineering, Mirpur University of Science & Technology, Mirpur AJK 10250, Pakistan
References (54)
Show source-provided references
1. Irtaza, A., Adnan, S.M., Ahmed, K.T., Jaffar, A., Khan, A., Javed, A., and Mahmood, M.T. (2018). An Ensemble Based Evolutionary Approach to the Class Imbalance Problem with Applications in CBIR. Appl. Sci., 8.10.3390/app8040495 ↗
2. Ye, J., Kobayashi, T., Toyama, N., Tsuda, H., and Murakawa, M. (2018). Acoustic Scene Classification Using Efficient Summary Statistics and Multiple Spectro-Temporal Descriptor Fusion. Appl. Sci., 8.10.3390/app8081363 ↗
3. Information fusion in content based image retrieval: A comprehensive overview10.1016/j.inffus.2017.01.003 ↗
4. Nazir, A., Ashraf, R., Hamdani, T., and Ali, N. (2018, January 3–4). Content based image retrieval system by using HSV color histogram, discrete wavelet transform and edge histogram descriptor. Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan.10.1109/ICOMET.2018.8346343 ↗
5. Unsupervised visual hashing with semantic assistant for content-based image retrieval10.1109/TKDE.2016.2562624 ↗
6. Semantic content-based image retrieval: A comprehensive study10.1016/j.jvcir.2015.07.012 ↗
7. Wan, J., Wang, D., Hoi, S.C.H., Wu, P., Zhu, J., Zhang, Y., and Li, J. (2014, January 3–7). Deep Learning for Content-Based Image Retrieval: A Comprehensive Study. Proceedings of the ACM International Conference on Multimedia, Orlando, FL, USA.10.1145/2647868.2654948 ↗
8. Ali, N., Bajwa, K.B., Sablatnig, R., Chatzichristofis, S.A., Iqbal, Z., Rashid, M., and Habib, H.A. (2016). A novel image retrieval based on visual words integration of SIFT and SURF. PLoS ONE, 11.10.1371/journal.pone.0157428 ↗
9. Image retrieval by addition of spatial information based on histograms of triangular regions10.1016/j.compeleceng.2016.04.002 ↗
10. O’Hara, S., and Draper, B.A. (arXiv, 2011). Introduction to the bag of features paradigm for image classification and retrieval, arXiv.
11. Sivic, J., and Zisserman, A. (2003, January 13–16). Video Google: A text retrieval approach to object matching in videos. Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France.10.1109/ICCV.2003.1238663 ↗
12. Adding spatial distribution clue to aggregated vector in image retrieval10.1186/s13640-018-0247-0 ↗
13. Ancient Coin Classification Using Reverse Motif Recognition: Image-based classification of Roman Republican coins10.1109/MSP.2015.2409331 ↗
14. Ali, N., Zafar, B., Riaz, F., Dar, S.H., Ratyal, N.I., Bajwa, K.B., Iqbal, M.K., and Sajid, M. (2018). A Hybrid Geometric Spatial Image Representation for scene classification. PLoS ONE, 13.10.1371/journal.pone.0203339 ↗
15. Intelligent Image Classification-Based on Spatial Weighted Histograms of Concentric Circles10.2298/CSIS180105025Z ↗
16. Lazebnik, S., Schmid, C., and Ponce, J. (2006, January 17–22). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA.
17. Li, X., Song, Y., Lu, Y., and Tian, Q. (December, January 28). Spatial pooling for transformation invariant image representation. Proceedings of the 19th ACM International Conference on Multimedia, Scottsdale, AZ, USA.
18. Karmakar, P., Teng, S.W., Lu, G., and Zhang, D. (2015, January 23–25). Rotation Invariant Spatial Pyramid Matching for Image Classification. Proceedings of the 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Adelaide, Australia.10.1109/DICTA.2015.7371303 ↗
19. Liu, D., Hua, G., Viola, P., and Chen, T. (2008, January 23–28). Integrated feature selection and higher-order spatial feature extraction for object categorization. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587403 ↗
20. Khan, R., Barat, C., Muselet, D., and Ducottet, C. (2012, January 3–7). Spatial orientations of visual word pairs to improve bag-of-visual-words model. Proceedings of the British Machine Vision Conference, Surrey, UK.10.5244/C.26.89 ↗
21. Zafar, B., Ashraf, R., Ali, N., Ahmed, M., Jabbar, S., and Chatzichristofis, S.A. (2018). Image classification by addition of spatial information based on histograms of orthogonal vectors. PLoS ONE, 13.10.1371/journal.pone.0198175 ↗
22. Fusion of local and global features for effective image extraction10.1007/s10489-017-0916-1 ↗
23. Wang, C., Zhang, B., Qin, Z., and Xiong, J. (2013). Spatial weighting for bag-of-features based image retrieval. Integrated Uncertainty in Knowledge Modelling and Decision Making, Springer.10.1007/978-3-642-39515-4_8 ↗
24. Image retrieval using spatiograms of colors quantized by gaussian mixture models10.1016/j.neucom.2015.07.008 ↗
25. Feature integration analysis of bag-of-features model for image retrieval10.1016/j.neucom.2012.08.061 ↗
26. Ali, N., Mazhar, D.A., Iqbal, Z., Ashraf, R., Ahmed, J., and Khan, F.Z. (arXiv, 2017). Content-Based Image Retrieval Based on Late Fusion of Binary and Local Descriptors, arXiv.10.20944/preprints201703.0156.v2 ↗
27. Filliat, D. (2007, January 10–14). A visual bag of words method for interactive qualitative localization and mapping. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy.10.1109/ROBOT.2007.364080 ↗
28. Real-time indoor assistive localization with mobile omnidirectional vision and cloud GPU acceleration10.3934/ElectrEng.2017.1.74 ↗
29. Exploiting global and local features for image retrieval10.1007/s11771-018-3735-6 ↗
30. Perceptual uniform descriptor and ranking on manifold for image retrieval10.1016/j.ins.2017.10.010 ↗
31. Image retrieval framework based on texton uniform descriptor and modified manifold ranking10.1016/j.jvcir.2017.08.002 ↗
32. Image retrieval based on non-uniform bins of color histogram and dual tree complex wavelet transform10.1007/s11042-016-3882-4 ↗
33. Deep learning based feature selection for remote sensing scene classification10.1109/LGRS.2015.2475299 ↗
34. Xia, G.S., Tong, X.Y., Hu, F., Zhong, Y., Datcu, M., and Zhang, L. (arXiv, 2017). Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation, arXiv.
35. Vassou, S.A., Anagnostopoulos, N., Amanatiadis, A., Christodoulou, K., and Chatzichristofis, S.A. (2017, January 19–21). Como: A compact composite moment-based descriptor for image retrieval. Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing, Florence, Italy.10.1145/3095713.3095744 ↗
36. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
37. Anwar, H., Zambanini, S., and Kampel, M. (2014, January 2–5). Encoding spatial arrangements of visual words for rotation-invariant image classification. Proceedings of the 36th German Conference, GCPR 2014, Münster, Germany.10.1007/978-3-319-11752-2_36 ↗
38. Tuytelaars, T. (2010, January 13–18). Dense interest points. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.10.1109/CVPR.2010.5539911 ↗
39. A novel image retrieval based on a combination of local and global histograms of visual words10.1155/2016/8217250 ↗
40. Real-time computerized annotation of pictures10.1109/TPAMI.2007.70847 ↗
41. Modeling the shape of the scene: A holistic representation of the spatial envelope10.1023/A:1011139631724 ↗
42. Zhou, J., Liu, X., Liu, W., and Gan, J. (2018). Image retrieval based on effective feature extraction and diffusion process. Multimedia Tools Appl., 1–28.10.1007/s11042-018-6192-1 ↗
43. Features for image retrieval: An experimental comparison10.1007/s10791-007-9039-3 ↗
44. Zhou, Z.H., and Xu, J.M. (2007, January 20–24). On the relation between multi-instance learning and semi-supervised learning. Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, USA.10.1145/1273496.1273643 ↗
45. Andrews, S., Tsochantaridis, I., and Hofmann, T. (2002). Support vector machines for multiple-instance learning. Advances in Neural Information Processing Systems, MIT Press.
46. Image Classification Using Fusion of Holistic Visual Descriptions10.5815/ijigsp.2016.08.07 ↗
47. A pooled Object Bank descriptor for image scene classification10.1016/j.eswa.2017.10.057 ↗
48. Yin, H. (2010). Scene Classification Using Spatial Pyramid Matching and Hierarchical Dirichlet Processes. [MSc Thesis, Rochester Institute of Technology].
49. Boosting local texture descriptors with Log-Gabor filters response for improved image retrieval10.1007/s13735-016-0099-2 ↗
50. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3–7). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, OR, Florida, USA.10.1145/2647868.2654889 ↗
51. Chatfield, K., Simonyan, K., Vedaldi, A., and Zisserman, A. (arXiv, 2014). Return of the devil in the details: Delving deep into convolutional nets, arXiv.10.5244/C.28.6 ↗
52. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
53. Mousavian, A., and Kosecka, J. (arXiv, 2015). Deep convolutional features for image based retrieval and scene categorization, arXiv.
54. Chathurani, N., Geva, S., Chandran, V., and Cynthujah, V. (2015, January 18–20). Content-Based Image (Object) Retrieval with Rotational Invariant Bag-of-Visual Words Representation. Proceedings of the 2015 IEEE 10th International Conference on Industrial and Information Systems (ICIIS), Peradeniya, Sri Lanka.10.1109/ICIINFS.2015.7399002 ↗