journal article·2019-02-20Open access

Comparative Evaluation of Hand-Crafted Image Descriptors vs. Off-the-Shelf CNN-Based Features for Colour Texture Classification under Ideal and Realistic Conditions

Raquel Bello-Cerezo, Francesco Bianconi, Francesco Di Maria, Paolo Napoletano, Fabrizio Smeraldi

Applied Sciences· Volume 9· Issue 4

Abstract

Convolutional Neural Networks (CNN) have brought spectacular improvements in several fields of machine vision including object, scene and face recognition. Nonetheless, the impact of this new paradigm on the classification of fine-grained images—such as colour textures—is still controversial. In this work, we evaluate the effectiveness of traditional, hand-crafted descriptors against off-the-shelf CNN-based features for the classification of different types of colour textures under a range of imaging conditions. The study covers 68 image descriptors (35 hand-crafted and 33 CNN-based) and 46 compilations of 23 colour texture datasets divided into 10 experimental conditions. On average, the results indicate a marked superiority of deep networks, particularly with non-stationary textures and in the presence of multiple changes in the acquisition conditions. By contrast, hand-crafted descriptors were better at discriminating stationary textures under steady imaging conditions and proved more robust than CNN-based features to image rotation.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Image and Video Retrieval Techniques · Image Retrieval and Classification Techniques · Visual Attention and Saliency Detection

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Engineering, Università degli Studi di Perugia, Via Goffredo Duranti 93–06125 Perugia (PG), Italy

Department of Informatics, Systems and Communication, Università degli studi di Milano-Bicocca, Viale Sarca 336–20125 Milano (MI), Italy

School of Electronic Engineering and Computer Science, Queen Mary University of London, 10 Godward Square, Mile End Road, London E1 4FZ, UK

Funding

Ministero dell’Istruzione, dell’Università e della Ricerca · Ministerio de Economía, Industria y Competitividad, Gobierno de España

References (97)

Show source-provided references
1. Local binary features for texture classification: Taxonomy and experimental study10.1016/j.patcog.2016.08.032 ↗
2. From BoW to CNN: Two Decades of Texture Representation for Texture Classification10.1007/s11263-018-1125-z ↗
3. Humeau-Heurtier, A. (2019). Texture feature extraction methods: A survey. IEEE Access.10.1109/ACCESS.2018.2890743 ↗
4. Krizhevsky, A., Sutskever, I., and Hinton, G. (2012, January 3–6). ImageNet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
5. Razavian, A., Azizpour, H., Sullivan, J., and Carlsson, S. (2014, January 23–28). CNN features off-the-shelf: An astounding baseline for recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA.10.1109/CVPRW.2014.131 ↗
6. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 5th International Conference on Learning Representations, San Diego, CA, USA.
7. Parkhi, O., Vedaldi, A., and Zissermann, A. (2015, January 7–10). Deep face recognition. Proceedings of the British Machine Vision Conference, Swansea, UK.10.5244/C.29.41 ↗
8. Nam, G.P., Choi, H., Cho, J., and Kim, I.J. (2018). PSI-CNN: A Pyramid-Based Scale-Invariant CNN Architecture for Face Recognition Robust to Various Image Resolutions. Appl. Sci., 8.10.3390/app8091561 ↗
9. Hertel, L., Barth, E., Kaster, T., and Martinetz, T. (2015, January 12–17). Deep convolutional neural networks as generic feature extractors. Proceedings of the International Joint Conference on Neural Networks, Killarney, Ireland.10.1109/IJCNN.2015.7280683 ↗
10. Classification with color and texture: Jointly or separately?10.1016/j.patcog.2003.11.011 ↗
11. Theoretical and experimental comparison of different approaches for color texture classification10.1117/1.3651210 ↗
12. Comparison of Texture Analysis Schemes Under Nonideal Conditions10.1109/TIP.2010.2101612 ↗
13. Choice of a pertinent color space for color texture characterization using parametric spectral analysis10.1016/j.patcog.2010.07.007 ↗
14. Influence of normalization and color space to color texture classification10.1016/j.patcog.2016.07.002 ↗
15. Cimpoi, M., Maji, S., and Vedaldi, A. (2015, January 7–12). Deep filter banks for texture recognition and segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7299007 ↗
16. Deep Filter Banks for Texture Recognition, Description, and Segmentation10.1007/s11263-015-0872-3 ↗
17. Combining multiple features for color texture classification10.1117/1.JEI.25.6.061410 ↗
18. Hand-Crafted vs. Learned Descriptors for Color Texture Classification10.1007/978-3-319-56010-6_22 ↗
19. Petrou, M., and García Sevilla, P. (2006). Image Processing. Dealing with Texture, Wiley Interscience.10.1002/047003534X ↗
20. Material-specific adaptation of color invariant features10.1016/j.patrec.2008.10.005 ↗
21. (2017, January 11). Amsterdam Library of Textures. Available online: http://aloi.science.uva.nl/public_alot/.
22. Brodatz, P. (1966). Textures: A Photographic Album for Artists and Designers, Dover.
23. (2017, January 11). Colored Brodatz Texture Database. Available online: http://multibandtexture.recherche.usherbrooke.ca/colored_brodatz.html.
24. Reflectance and Texture of Real-World Surfaces10.1145/300776.300778 ↗
25. (2017, January 25). CUReT: Columbia-Utrecht Reflectance and Texture Database. Available online: http://www.cs.columbia.edu/CAVE/software/curet/index.php.
26. Visual Geometry Group (2017, January 26). CUReT: Columbia-Utrecht Reflectance and Texture Database. Available online: http://www.robots.ox.ac.uk/~vgg/research/texclass/setup.html.
27. The scale of geometric texture10.1007/978-3-642-33718-5_5 ↗
28. (2017, January 11). Drexel Texture Database. Available online: https://www.cs.drexel.edu/~kon/texture/.
29. Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. (2014, January 23–28). Describing textures in the wild. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.461 ↗
30. (2019, January 19). Describable Texture Database (DTD). Available online: https://www.robots.ox.ac.uk/~vgg/data/dtd/.
31. Fine-grained material classification using micro-geometry and reflectance10.1007/978-3-319-46454-1_47 ↗
32. (2017, January 11). The Fabrics Dataset. Available online: http://ibug.doc.ic.ac.uk/resources/fabrics/.
33. A database for automatic classification of forest species10.1007/s00138-012-0417-5 ↗
34. (2017, January 11). ForestSpecies Database. Available online: http://web.inf.ufpr.br/vri/image-and-videos-databases/forest-species-database.
35. Hayman, E., Caputo, B., Fritz, M., and Eklundh, J. (2002, January 11–14). On the significance of real-world conditions for material classification. Proceedings of the 8th European Conference on Computer Vision (ECCV 2004), Prague, Czech Republic.10.1007/978-3-540-24673-2_21 ↗
36. (2017, January 11). The KTH-TIPS and KTH-TIPS2 Image Databases. Available online: http://www.nada.kth.se/cvap/databases/kth-tips/download.html.
37. Caputo, B., Hayman, E., and Mallikarjuna, P. (2005, January 17–20). Class-specific material categorisation. Proceedings of the 10th International Conference on Computer Vision (ICCV), Beijing, China.10.1109/ICCV.2005.54 ↗
38. Kylberg, G. (2014). Automatic Virus Identification Using TEM. Image Segmentation and Texture Analysis. [Ph.D. Thesis, Faculty of Science and Technology, University of Uppsala].
39. (2017, January 11). Kylberg Sintorn Rotation Dataset. Available online: http://www.cb.uu.se/~gustaf/KylbergSintornRotation/.
40. Multimodal Feature-Based Surface Material Classification10.1109/TOH.2016.2625787 ↗
41. (2017, January 25). LMT Haptic Texture Database. Available online: http://www.lmt.ei.tum.de/downloads/texture/.
42. On comparing colour spaces from a performance perspective: application to automated classification of polished natural stones
43. (2019, January 19). MondialMarmi: A Collection of Images of Polished Natural Stones for Colour and Texture Analysis, v2.0. Available online: http://dismac.dii.unipg.it/mm/ver_2_0/index.html.
44. New Brodatz-based Image Databases for Grayscale Color and Multiband Texture Analysis
45. (2017, January 12). Multiband Texture Database. Available online: http://multibandtexture.recherche.usherbrooke.ca/multi_band_more.html.
46. (2017, February 23). Normalized Brodatz’s Texture Database. Available online: http://multibandtexture.recherche.usherbrooke.ca/normalized_brodatz.html.
47. A new benchmark image test suite for evaluating color texture classification schemes10.1007/s11042-013-1418-8 ↗
48. (2017, January 12). New [BarkTex] Benchmark Image Test Suite for Evaluating Color Texture Classification Schemes. Available online: https://www-lisic.univ-littoral.fr/~porebski/BarkTex_image_test_suite.html.
49. Classification of color textures by Gabor filtering
50. Outex—New Framework for Empirical Evaluation of Texture Analysis Algorithms10.1109/ICPR.2002.1044854 ↗
51. (2018, January 12). Outex Texture Database. Available online: http://www.outex.oulu.fi/.
52. Performance analysis of colour descriptors for parquet sorting10.1016/j.eswa.2012.09.007 ↗
53. (2017, January 12). Parquet Dataset. Available online: http://dismac.dii.unipg.it/parquet/index.html.
54. Plant leaf identification using Gabor wavelets10.1002/ima.20201 ↗
55. (2019, January 11). 1200Tex Dataset. Available online: https://scg.ifsc.usp.br/dataset/1200Tex.php/.
56. Bormann, R., Esslinger, D., Hundsdörfer, D., Högele, M., and Vincze, M. (2016, January 21–22). Texture characterization with semantic attributes: Database and algorithm. Proceedings of the 47th International Symposium on Robotics (ISR 2016), Munich, Germany.
57. Evaluating color texture descriptors under large variations of controlled lighting conditions10.1364/JOSAA.33.000017 ↗
58. (2017, January 12). RawFooT DB: Raw Food Texture Database. Available online: http://projects.ivl.disco.unimib.it/minisites/rawfoot/.
59. (2017, January 12). Salzburg texture Image Database (STex). Available online: http://www.wavelab.at/sources/STex/.
60. Color texture analysis based on fractal descriptors10.1016/j.patcog.2011.11.009 ↗
61. (2019, January 19). USPTex Database. Available online: http://fractal.ifsc.usp.br/dataset/USPtex.php.
62. (2017, January 12). VisTex Reference Textures. Available online: http://vismod.media.mit.edu/vismod/imagery/VisionTexture/Images/Reference/.
63. Performance evaluation of soft color texture descriptors for surface grading using experimental design and logistic regression10.1016/j.patcog.2007.09.011 ↗
64. Experimental comparison of color spaces for material classification10.1117/1.JEI.25.6.061406 ↗
65. Pietikäinen, M., Nieminen, S., Marszalec, E., and Ojala, T. (1996, January 25–29). Accurate Color Discrimination with Classification Based on Features Distributions. Proceedings of the 13th International Conference on Pattern Recognition (ICPR’96), Vienna, Austria.10.1109/ICPR.1996.547285 ↗
66. Color Indexing10.1007/BF00130487 ↗
67. Texture description through histograms of equivalent patterns10.1007/s10851-012-0349-8 ↗
68. A Completed Modeling of Local Binary Pattern Operator for Texture Classification10.1109/TIP.2010.2044957 ↗
69. He, Y., and Sang, N. (2011, January 10–12). Robust Illumination Invariant Texture Classification Using Gradient Local Binary Patterns. Proceedings of the 2011 International Workshop on Multi-Platform/Multi-Sensor Remote Sensing and Mapping, Xiamen, China.10.1109/M2RSM.2011.5697422 ↗
70. Jin, H., Liu, Q., Lu, H., and Tong, X. (2004, January 18–20). Face detection using improved LBP under Bayesian framework. Proceedings of the 3rd International Conference on Image and Graphics, Hong Kong, China.
71. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns10.1109/TPAMI.2002.1017623 ↗
72. Enhanced local texture feature sets for face recognition under difficult lighting conditions10.1109/TIP.2010.2042645 ↗
73. Texture Unit, Texture Spectrum, And Texture Analysis10.1109/TGRS.1990.572934 ↗
74. An investigation on the use of local multi-resolution patterns for image classification
75. Texture classification using rotation invariant LBP based on digital polygons
76. Textural Features for Image Classification10.1109/TSMC.1973.4309314 ↗
77. Rotation invariant co-occurrence features based on digital circles and discrete Fourier transform10.1016/j.patrec.2014.04.006 ↗
78. Dalal, N., and Triggs, B. (2005, January 20–26). Histograms of oriented gradients for human detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), San Diego, CA, USA.
79. A statistical approach to material classification using image patch exemplars10.1109/TPAMI.2008.182 ↗
80. Rotation-invariant texture classification10.1016/S0167-8655(02)00285-4 ↗
81. Texture classification using wavelet transform10.1016/S0167-8655(02)00390-2 ↗
82. A Statistical Approach to Texture Classification from Single Images10.1007/s11263-005-4635-4 ↗
83. Generalization of the cooccurrence matrix for colour images: Application to colour texture classification10.5566/ias.v23.p63-72 ↗
84. Color texture classification by integrative Co-occurrence matrices10.1016/j.patcog.2003.09.010 ↗
85. Combining local binary patterns and local color contrast for texture classification under varying illumination10.1364/JOSAA.31.001453 ↗
86. Local color vector binary patterns from multichannel face images for face recognition10.1109/TIP.2011.2181526 ↗
87. Chen, C.H., and Wang, P.S.P. (2005). Texture Analysis with Local Binary Patterns. Handbook of Pattern Recognition and Computer Vision, World Scientific Publishing. [3rd ed.].10.1142/5711 ↗
88. Improved opponent color local binary patterns: An effective local image descriptor for color texture classification
89. A Multiscale Representation Including Opponent Color Features for Texture Recognition10.1109/83.650858 ↗
90. Huang, G., Liu, Z., and Weinberger, K.Q. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
91. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
92. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
93. Chatfield, K., Simonyan, K., Vedaldi, A., and Zisserman, A. (2014, January 1–5). Return of the devil in the details: Delving deep into convolutional nets. Proceedings of the British Machine Vision Conference 2014, Nottingham, UK.10.5244/C.28.6 ↗
94. A unifying framework for LBP and related methods
95. Aggregating local image descriptors into compact codes10.1109/TPAMI.2011.235 ↗
96. Texture King (2016, December 14). Free Stock Textures, TextureKing. Available online: http://www.textureking.com.
97. Kanji, G. (2006). 100 Statistical Tests, SAGE Publications Ltd.. [3rd ed.].10.4135/9781849208499 ↗