Image Retrieval Method Based on Image Feature Fusion and Discrete Cosine Transform
Source abstract
Abstract
This paper presents a new content-based image retrieval (CBIR) method based on image feature fusion. The deep features are extracted from object-centric and place-centric deep networks. The discrete cosine transform (DCT) solves the strong correlation of deep features and reduces dimensions. The shallow features are extracted from a Quantized Uniform Local Binary Pattern (ULBP), hue-saturation-value (HSV) histogram, and dual-tree complex wavelet transform (DTCWT). Singular value decomposition (SVD) is applied to reduce the dimensions of ULBP and DTCWT features. The experimental results tested on Corel datasets and the Oxford building dataset show that the proposed method based on shallow features fusion can significantly improve performance compared to using a single type of shallow feature. The proposed method based on deep features fusion can slightly improve performance compared to using a single type of deep feature. This paper also tests variable factors that affect image retrieval performance, such as using principal component analysis (PCA) instead of DCT. The DCT can be used for dimensional feature reduction without losing too much performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Retrieval and Classification Techniques · Advanced Image and Video Retrieval Techniques · Remote-Sensing Image Classification
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Technology, Anhui University of Finance and Economics, Bengbu 233000, China
Department of Electronics Engineering, Sangmyung University, Seoul 03016, Korea
Funding
National Research Foundation of Korea
References (76)
Show source-provided references
1. Text-based, content-based, and semantic-based image retrievals: A survey
2. CBSA: Content-based soft annotation for multimodal image retrieval using bayes point machines10.1109/TCSVT.2002.808079 ↗
3. Intelligent Image Retrieval Techniques: A Survey10.1016/S1665-6423(14)71609-8 ↗
4. Tkalcic, M., and Tasic, J.F. (2003, January 22–24). Colour spaces: Perceptual, historical and applicational background. Proceedings of the EUROCON 2003, Ljubljana, Slovenia.
5. Sural, S., Qian, G., and Pramanik, S. (2002, January 22–25). Segmentation and histogram generation using the HSV color space for image retrieval. Proceedings of the International Conference on Image Processing, Rochester, NY, USA.
6. Modified color motif co-occurrence matrix for image indexing and retrieval10.1016/j.compeleceng.2012.11.023 ↗
7. Pass, G., Zabih, R., and Miller, J. (1996, January 18–22). Comparing Images Using Color Coherence Vectors. Proceedings of the Fourth ACM International Conference on Multimedia, Boston, MA, USA.10.1145/244130.244148 ↗
8. Ahonen, T., Hadid, A., and Pietikäinen, M. (2004, January 11–14). Face recognition with local binary patterns. Proceedings of the European Conference on Computer Vision, Prague, Czech Republic.10.1007/978-3-540-24670-1_36 ↗
9. Local Tetra Patterns: A New Feature Descriptor for Content-Based Image Retrieval10.1109/TIP.2012.2188809 ↗
10. Directional local extrema patterns: A new descriptor for content based image retrieval10.1007/s13735-012-0008-2 ↗
11. Color texture image retrieval based on Gaussian copula models of Gabor wavelets10.1016/j.patcog.2016.10.030 ↗
12. Nazir, A., Ashraf, R., Hamdani, T., and Ali, N. (2018, January 3–4). Content based image retrieval system by using HSV color histogram, discrete wavelet transform and edge histogram descriptor. Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan.10.1109/ICOMET.2018.8346343 ↗
13. Vetova, S., and Ivanov, I. (2015, January 17–19). Comparative analysis between two search algorithms using DTCWT for content-based image retrieval. Proceedings of the 3rd International Conference on Circuits, Systems, Communications, Computers and Applications, Seville, Spain.10.1109/ICIT.2015.7125336 ↗
14. A New Approach to Region Based Image Retrieval using Shape Adaptive Discrete Wavelet Transform10.5815/ijigsp.2016.01.01 ↗
15. Yang, M., Kpalma, K., and Ronsin, J. (2012). Shape-based invariant feature extraction for object recognition. Advances in Reasoning-Based Image Processing Intelligent Systems, Springer.10.1007/978-3-642-24693-7_9 ↗
16. Shape-based image retrieval using generic Fourier descriptor10.1016/S0923-5965(02)00084-X ↗
17. Fusion of local and global features for effective image extraction10.1007/s10489-017-0916-1 ↗
18. Distinctive Image Features from Scale-Invariant Keypoints10.1023/B:VISI.0000029664.99615.94 ↗
19. Bay, H., Tuytelaars, T., and Van Gool, L. (2006, January 7–13). Surf: Speeded up robust features. Proceedings of the European Conference on Computer Vision, Graz, Austria.10.1007/11744023_32 ↗
20. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of oriented gradients for human detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
21. Torralba, A., Murphy, K.P., Freeman, W.T., and Rubin, M.A. (2003, January 13–16). Context-based vision system for place and object recognition. Proceedings of the IEEE International Conference on Computer Vision, Nice, France.10.1109/ICCV.2003.1238354 ↗
22. Image indexing and retrieval in JPEG compressed domain based on vector quantization10.1016/j.mcm.2011.11.064 ↗
23. Arandjelović, R., and Zisserman, A. (2012, January 16–21). Three things everyone should know to improve object retrieval. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248018 ↗
24. Content-based image retrieval using features extracted from halftoning-based block truncation coding
25. Efficient Visual Search of Videos Cast as Text Retrieval10.1109/TPAMI.2008.111 ↗
26. Arandjelovic, R., and Zisserman, A. (2013, January 23–28). All about VLAD. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.207 ↗
27. Content Based Image Retrieval Using Embedded Neural Networks with Bandletized Regions10.3390/e17063552 ↗
28. Image Retrieval Using Fused Deep Convolutional Features10.1016/j.procs.2017.03.159 ↗
29. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
30. Kanwal, K., Ahmad, K.T., Khan, R., Abbasi, A.T., and Li, J. (2020). Deep Learning Using Symmetry, FAST Scores, Shape-Based Filtering and Spatial Mapping Integrated with CNN for Large Scale Image Retrieval. Symmetry, 12.10.3390/sym12040612 ↗
31. Cao, Z., Shaomin, M.U., Yongyu, X.U., and Dong, M. (2018, January 14–17). Image retrieval method based on CNN and dimension reduction. Proceedings of the 2018 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Jinan, China.10.1109/SPAC46244.2018.8965601 ↗
32. A new matching strategy for content based image retrieval system10.1016/j.asoc.2013.10.003 ↗
33. A new fusion approach for content based image retrieval with color histogram and local directional pattern10.1007/s13042-016-0597-9 ↗
34. Content based image retrieval using image features information fusion10.1016/j.inffus.2018.11.004 ↗
35. Correlated microstructure descriptor for image retrieval10.1109/ACCESS.2019.2911954 ↗
36. A novel image retrieval method based on multi-features fusion10.1016/j.image.2020.115911 ↗
37. An efficient image retrieval framework using fused information feature10.1016/j.compeleceng.2019.01.022 ↗
38. Yu, H., Li, M., Zhang, H.J., and Feng, J. (2002, January 22–25). Color texture moments for content-based image retrieval. Proceedings of the International Conference on Image Processing, Rochester, NY, USA.
39. Discrete Cosine Transform10.1109/T-C.1974.223784 ↗
40. Singular value decomposition for genome-wide expression data processing and modeling10.1073/pnas.97.18.10101 ↗
41. Texture Image Retrieval Using DTCWT-SVD and Local Binary Pattern Features
42. Loesdau, M., Chabrier, S., and Gabillon, A. (2014). Hue and saturation in the RGB color space. International Conference on Image and Signal Processing, Springer.10.1007/978-3-319-07998-1_23 ↗
43. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns10.1109/TPAMI.2002.1017623 ↗
44. The dual-tree complex wavelet transform10.1109/MSP.2005.1550194 ↗
45. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
46. Loeffler, C., Ligtenberg, A., and Moschytz, G. (1989, January 23–26). Practical, fast 1-D DCT algorithms with 11 multiplications. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Glasgow, UK.
47. Evaluation of image similarity by histogram intersection10.1002/col.20122 ↗
48. Liu, L., Fieguth, P., Wang, X., Pietikäinen, M., and Hu, D. (2016, January 11–14). Evaluation of LBP and deep texture descriptors with a new robustness benchmark. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46487-9_5 ↗
49. Modified image retrieval algorithm based on DT-CWT
50. Artificial Intelligence in China10.1109/MIS.2008.99 ↗
51. Rat Brain Registration Using Improved Speeded Up Robust Features10.1007/s40846-016-0204-2 ↗
52. Visual Odometry Implementation and Accuracy Evaluation Based on Real-time Appearance-based Mapping10.18494/SAM.2020.2870 ↗
53. Places: A 10 Million Image Database for Scene Recognition10.1109/TPAMI.2017.2723009 ↗
54. Jiang, D., and Kim, J. (2018). Video searching and fingerprint detection by using the image query and PlaceNet-based shot boundary detection method. Appl. Sci., 8.10.3390/app8101735 ↗
55. MathWorks Deep Learning Toolbox Team (2021, June 19). Deep Learning Toolbox Model for ResNet-50 Network. Available online: https://www.mathworks.com/matlabcentral/fileexchange/?q=profileid:8743315.
56. Checinski, K., and Wawrzynski, P. (2020). DCT-Conv: Coding filters in convolutional networks with Discrete Cosine Transform. arXiv.10.1109/IJCNN48605.2020.9207103 ↗
57. (2021, June 19). Cosine Similarity. Available online: https://en.wikipedia.org/wiki/Cosine_similarity.
58. Tabak, J. (2014). Geometry: The Language of Space and Form, Infobase Publishing.
59. Wikipedia (2021, June 19). Manhattan Distance. Available online: https://en.wikipedia.org/wiki/Taxicab_geometry.
60. Wikipedia (2021, June 19). Minkowski Distance. Available online: https://en.wikipedia.org/wiki/Minkowski_distance.
61. Jurman, G., Riccadonna, S., Visintainer, R., and Furlanello, C. (2009, January 11). Canberra distance on ranked lists. Proceedings of the Advances in Ranking NIPS 09 Workshop, Whistler, BC, Canada.
62. Wikipedia (2021, June 19). Precision and Recall. Available online: https://en.wikipedia.org/wiki/Precision_and_recall.
63. SIMPLIcity: Semantics-sensitive integrated matching for picture libraries10.1109/34.955109 ↗
64. Image retrieval based on micro-structure descriptor10.1016/j.patcog.2011.02.003 ↗
65. Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval10.1109/TIP.2009.2035223 ↗
66. Philbin, J., Chum, O., Isard, M., Sivic, J., and Zisserman, A. (2007, January 17–22). Object retrieval with large vocabularies and fast spatial matching. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383172 ↗
67. Image retrieval based on multi-texton histogram10.1016/j.patcog.2010.02.012 ↗
68. Content-based image retrieval using color difference histogram10.1016/j.patcog.2012.06.001 ↗
69. A novel image retrieval method based on hybrid information descriptors10.1016/j.jvcir.2014.06.016 ↗
70. Correlated primary visual texton histogram features for content base image retrieval10.1109/ACCESS.2018.2866091 ↗
71. (2021, June 19). C++ Code to Compute the Ground Truth. Available online: https://www.robots.ox.ac.uk/~vgg/data/oxbuildings/compute_ap.cpp.
72. Tsai, C.Y., Lin, T.C., Wei, C.P., and Wang, Y.C.F. (2014, January 4–9). Extended-bag-of-features for translation, rotation, and scale-invariant image retrieval. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854932 ↗
73. On Lines and Planes of Closest Fit to Systems of Points in Space10.1080/14786440109462720 ↗
74. Large scale category-structured image retrieval for object identification through supervised learning of CNN and SURF-based matching10.1109/ACCESS.2020.2982560 ↗
75. Liu, H., Wang, R., Shan, S., and Chen, X. (2016, January 27–30). Deep supervised hashing for fast image retrieval. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.227 ↗
76. Jégou, H., Douze, M., and Schmid, C. (2009, January 20–25). On the burstiness of visual elements. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPRW.2009.5206609 ↗